karma expect

This project is based on James Shore: Let's Code: Test Driven Javascript lessons (Chapter 7, Cross-Browser Testing). We will set up a client side testing framework using Grunt as task-runner, Karma as test-runner, Mocha as testing framework and Expect as an assertion library.

Karma is a test runner. You can test your javascript code on many browsers at the same time. The test process can be automated, and it can be run automatically as your code changing. As its home page says

Karma is essentially a tool which spawns a web server that executes source code against test code for each of the browsers connected. The results for each test against each browser are examined and displayed via the command line to the developer such that they can see which browsers and tests passed or failed.

Karma runs on Node.js. If you are working on a Windows machine, or you have not used Node.js yet, this post may help you try out these tools. Ok, so, let's install Karma.

(we suppose that you installed Node.js and you have a package.json file in your working directory)

After installation make a script which can launch Karma:

Running Karma without any parameter we get this message:

Now we can initialize Karma:

We need to install Mocha (or other test framework) to use Karma. Besides Mocha, we install karma-mocha , an adapter for Mocha:

Now we can start Karma server:

As you can see Karma server started at http://localhost:9876/. I you run a browser, and type this url, the Karma connects to the browser:

You can connect as many browsers as you want (check out the available browsers here). If you want to launch a browser from Virtualbox (e.g. IE8 on Windows), here are some good tips about how to address localhost.

We have Karma server, but nothing to test yet. Let's install karma-expect , the Expect.js adapter first:

Then we have to add expect to frameworks in the Karma config file ( karma.conf.js ):

Let's write a simple test that compares two strings:

Now start again the Karma server (because we have modified its config file just now), and run the test in an other terminal (and make sure you connected at least one browser to Karma):

We are going to use Grunt for task automation. The work flow is the following:

  • start Karma server
  • launch the browsers manually (if you use virtual machine for some exotic browsers, start the machine and launch those browsers there)
  • connect the browsers to the Karma server (see above)
  • start grunt
  • grunt runs Karma to test your code, and checks if the tests pass or fail

First you need Grunt. After installing Grunt let's run it:

We have to create a simple Gruntfile and configure Karma (or if Gruntfile exists, add Karma to this). We'll accomplish this with Node.js child_process.exec() :

Grunt works synchronously, but it can be switched to asynchronous by calling this.async() within the task body (for more information see Grunt documentation).

You may want to get a feedback if the tests passed. According to the Node.js documentation the error argument - getting by the callback - will be null on success. All we need to do is passing false to the done() method to signal at Grunt if the tests failed.

If we start Karma server, launch our browsers and connect them, and run Grunt, then we can see the following:

And now only with Chrome, let's see what happens if the test fails:

Our aim was to set up a simple cross-browser testing framework, using Karma. We needed a test framework - it was Mocha - and some assertion library (I chose Expect.js). First we installed Karma and configured it, then installed Mocha and Expect.js adapter for Karma. After installations we were able to start Karma server and connect browsers to it. We wrote a simple test, ad ran it with Karma. We wanted to automate the test-running, using Grunt, the task runner. We configured a task that can launch Karma as a child process, and check if the tests pass or fail.

As I was working on setting up the framework, I tried grunt-karma , a Karma runner plugin for Grunt. It is really useful, you can simply configure Karma with this. After installing additional browser-launcher plugins, you don't need to start Karma and connect the browser manually, Grunt does the work, it starts the server, launches the given browsers, run the tests, closes the browsers and the Karma server (Single Run mode). Beyond that you can set - whenever your test files are being modified - to run Karma automatically. But I had difficulties when I tried to connect browsers launched in virtual machine (I need vm to run different kinds of Internet Explorers). The Karma runner couldn't see that browsers. I tried 'node-virtualbox', and a lot of plugins that works with virtual-machines, but I couldn't tune up them to work with grunt-karma plugin. After experimenting with grunt-karma , I left it and figured out a work flow, that needs to start the Karma server and the given browsers manually, but the testing can be automated by Grunt.


Unit Testing React.js With Jasmine and Karma

I have been mentioning React.js several times before. Recently I had opportunity to play with it closely.

I also wanted to make it right from the beginning. I wanted to Unit tests.

My choice of framework is Jasmine

Quick search for working example gave only this


Code Coverage of Jasmine Tests using Istanbul and Karma

For modern web application development, having dozens of unit tests is not enough anymore. The actual code coverage of those tests would reveal if the application is thoroughly stressed or not. For tests written using the famous Jasmine test library, an easy way to have the coverage report is via Istanbul and Karma.

For this example, let’s assume that we have a simple library sqrt.js which contains an alternative implementation of Math.sqrt . Note also how it will throw an exception instead of returning NaN for an invalid input.

Using Jasmine placed under test/lib/jasmine-1.3.1 , we can craft a test runner that includes the following spec:

Opening the spec runner in a web browser will give the expected outcome:

Karma expect

So far so good. Now let’s see how the code coverage of our test setup can be measured.

The first order of business is to install Karma. If you are not familiar with Karma, it is basically a test runner which can launch and connect to a specific set of web browsers, run your tests, and then gather the report. Using Node.js, what we need to do is:

Before launching Karma, we need to specify its configuration. It could be as simple as the following my.conf.js (most entries are self-explained). Note that the tests are executed using PhantomJS for simplicity, it is however quite trivial to add other web browsers such as Chrome and Firefox.

Running the tests, as well as performing code coverage at the same time, can be triggered via:

which will dump the output like:

As expected (from the previous manual invocation of the spec runner), the test passed just fine. However, the most particular interesting piece here is the code coverage report, it is stored (in the default location) under the subdirectory coverage . Open the report in your favorite browser and there you’ll find the coverage analysis report.

Karma expect

Behind the scene, Karma is using Istanbul, a comprehensive JavaScript code coverage tool (read also my previous blog post on JavaScript Code Coverage with Istanbul). Istanbul parses the source file, in this example sqrt.js , using Esprima and then adds some extra instrumentation which will be used to gather the execution statistics. The above report that you see is one of the possible outputs, Istanbul can also generate LCOV report which is suitable for many continuous integration systems (Jenkins, TeamCity, etc). An extensive analysis of the coverage data should also prevent any future coverage regression, check out my other post Hard Thresholds on JavaScript Code Coverage.

One important thing about code coverage is branch coverage. If you pay attention carefully, our test above is still not exercising the situation where the input to My.sqrt is negative. There is a big “I” marking in the third-line of the code, this is Istanbul telling us that the if branch is not taken at all (for the else branch, it will be an “E” marker). Once this missing branch is noticed, improving the situation is as easy as adding one more test to the spec:

Once the test is executed again, the code coverage report looks way better and everyone is happy.


Unit Testing with Karma and Jasmine in Angular 2

Karma expect

Angular 2 provides multiple options to support Unit Testing. In here, we will cover how to setup Jasmine, Karma and write some common unit tests for your Angular 2 Applications.

  • Preparing the package.json for new libraries
  • Introduction to Jasmine and Karma
  • Setting up Karma
  • The first karma test
  • Testing Angular 2 Components with Jasmine

To get to the fun as quickly as possible, we can start a new project following the instructions in the QuickStart. And make sure The project folder structure looks like this:

Preparing the package.json for new libraries

Modify the “scripts” property in the package.json as follows:

We’ve included a number of npm scripts in our suggested package.json to handle common development tasks. Here’s what these scripts do:

  • npm start – runs the compiler and a server at the same time, both in “watch mode”
  • npm run lite – runs the lite-server, a light-weight, static file server with excellent support for Angular apps that use routing.
  • npm run postinstall – called by npm automatically after it successfully completes package installation. This script installs the TypeScript definition files defined in typings.json.
  • npm run test – compiles typescript files, runs unit tests and then open the generated report
  • npm run tsc – runs the TypeScript compiler once
  • npm run tsc:w – runs the TypeScript compiler in watch mode; the process keeps running, awaiting changes to TypeScript files and recompiling when it sees them
  • npm run typings – runs the typings tool separately

Also, we need to install karma and all other dependencies. We will modify the “devDependencies” property as follows:

Using npm from the command line, install the packages listed in package.json with the command:

Using a terminal window, we will be needed to change into the directory that we are working before using the commands

Error messages—in red—might appear during the install, and you might see npm WARN messages. As long as there are no npm ERR! messages at the end, you can assume success.

Introduction to Jasmine and Karma

Jasmine is an open source testing framework from Pivotal Labs that uses behaviour-driven notation resulting in a fluent and improved testing experience.

  • Suites — describe(string, function) functions, take a title and a function containing one or more specs.
  • Specs — it(string, function) functions, take a title and a function containing one or more expectations.
  • Expectations — are assertions that evaluate to true or false. Basic syntax reads expect(actual).toBe(expected)
  • Matchers — are predefined helpers for common assertions. Eg: toBe(expected), toEqual(expected). Find a complete list here.

The karma test runner is ideal for writing and running unit tests while developing the application. It can be an integral part of the project’s development and continuous integration processes. This chapter describes how to setup and run tests with karma.

With Angular 2, Karma needs a kind of initialization file which diferentiate our app files from our test files and it loads the systemjs.config.js file as part of that process.. We will be added a file named karma-test-shim.js in our root directory as below:

Karma also needs a configuration file which sets which files should be loaded and what browsers should be used for testing. This new file called karma.conf.js will be added to our root directory:

We’ll start with a simple test to make sure the setup works properly. It’s often better to create an appropriate folder for them in the tests directory. So we’ll create a new file called 1st.spec.ts in the app/tests folder.

Tests written in Jasmine are called specs. The filename extension must be .spec.ts, the convention adhered to by karma.conf.js and other tooling.

Put spec files somewhere within the app/ folder. The karma.conf.js tells karma to look for spec files there

Add the following code to app/tests/1st.spec.ts:

Compile and run it in karma from the command line.

After a few moments, karma opens a browser and starts writing to the console.

Also, we can view debug mode by click the “DEBUG” button; it opens a new browser tab and re-runs the tests.

That command first compiles the application, then simultaneously re-compiles and runs the karma test-runner. Both the compiler and the karma watch for (different) file changes.

Shut it down manually with Ctrl-C.

Testing Angular 2 Components with Jasmine

We’ll add the file named app/app.component.spec.ts sits in the same folder as the component.

Start with ES6 import statements to get access to symbols referenced in the spec:

Here’s the setup for the tests followed by observations about the beforeEach:

Do not configure the TestBed after calling createComponent.

Now lets re-compile and run it in karma from the command line and see the changes from the view debug mode.

Test-runner output appears in the terminal window. We can update our app and our tests in real-time, keeping a weather eye on the console for broken tests. Karma is occasionally confused and it is often necessary to shut down its browser or even shut the command down (Ctrl-C) and restart it. No worries; it’s pretty quick.

let’s verify we have the following structure:

The repository for this project is available at here.


RunKit + Karma expect

This is a playground to test code. It runs a full Node.js environment and already has all of npm ’s 400,000 packages pre-installed , including karma-expect-maptalks with all npm packages installed . Try it out :

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.