Karate Home Page​ | Original Google Drive ​Link​ for this document | Please send corrections and feedback

to ​@ptrthomas
REST-assured Karate References / Comments
1 BDD Syntax Yes Yes
2 True DSL No. Fluent Interface. Yes DSL vs Fluent Interface.
Also IDE formatting is a ​challenge Also see (24) and (25)
3 Runs on the JVM Yes Yes
4 Implementation Java and Groovy Java
5 Code-base Size Large. Medium. Karate has API-mocks,
perf-testing and UI
40,000 lines of code (source: ​OpenHub​) 30,000 lines of code (source: ​OpenHub​) automation as well.

6 Mature Yes. Inception 2010. Yes. Inception February 2017. 2000+ GitHub “stars” in 2
years
Lots of blog posts, tutorials and Clear signs of ​wide and rapid adoption​.
StackOverflow posts. The quality of documentation
In the ​ThoughtWorks Tech Radar​ within and examples is arguably ​way
In the ThoughtWorks Tech Radar ​at 5 years​. just 2 years. better for Karate.

Active community on ​Stack Overflow​.

7 JsonPath Groovy GPath JayWay JsonPath GPath has some ​limitations
Implementation and ​updates are not possible
8 XPath implementation Groovy GPath and “XMLSlurper”. Standard W3C​ standard XPath using the Java
XPath is also supported, but paths that built-in XML lib. You can even update
return XML nodes cannot be used in XML documents using XPath.
assertions. Updating an XML document is
not possible.
9 HTTP Client Apache 4.X, but the code depends on Pluggable (future-proof). From v0.3 Karate even has an option to
deprecated APIs​. onwards, both the Apache and Jersey mock a servlet container
HTTP clients are supported. because of this abstraction.
There are ​some concerns​ with this d
​ esign​.
This means that you won’t be blocked if Karate also has ​minimal
More details in this ​issue​. your project already has a conflicting maven dependencies​.
version of one of these.
10 Quick Start / Project No Yes (Maven Archetype). Dev onboarding experience
Scaffolding much better with Karate.
There’s also a ​standalone executable​. Archetype Includes working
example.
11 Test-Scripting Java Karate-Script (Cucumber / Gherkin) No Java knowledge needed for
Language Karate
12 Test Scripts have to be Yes No Tests are plain-text. No IDE
compiled required for Karate
13 IDE Support Yes. Intelli-Sense, Auto-Complete and Partial. Eclipse and IntelliJ have Since you can re-use JSON
Refactoring work for Java and POJO-s Cucumber plugins that work well and payloads across tests, the
have pretty good syntax coloring. “re-factorability” aspect is
covered as well.
Not needing POJO-s means that the lines
of code required for a test is ​dramatically
reduced​, see (39).
14 Step Through / Yes. Java + IDE Support. Even better ! Debug in Visual Studio Also see (42)
Debug-ability Code with​ step-back and ​even​ hot-reload
(since v0.9.5).

also see the ​Dev-mode HTML report​:

steps, error diagnostics and HTTP logs
in-line - since v0.6.2
15 Test Runner Any, bring your own. TestNG or JUnit will JUnit supported and can co-exist with And Karate’s parallel
work. TestNG in the same project if needed. execution capability is in
“core”, independent of even
Maven or any unit-testing
framework.
16 Tags / Groups Built In No (have to use TestNG or equivalent) Yes

17 Extend with custom Java code JavaScript No need to compile, and

routines via... easier for non-programmers.
18 Re-use Java code Yes Yes (via JavaScript interop)
19 Validate All Payload Not built-in. You ​need​ to use external Karate natively supports a “​deep equals”​ IMO the ​biggest​ limitation of
values in one step libraries such as Hamcrest. and “​contains in any order​” assertion for REST-Assured:
JSON, JSON arrays and XML, ​and​ lets you ● Example 1
● No easy way to do a “deep equals” ignore chosen fields at the same time. ● Example 2
comparison for nested objects. ● Example 3
● No way to “ignore” fields - for e.g. id / This is super-important for ​GraphQL​. ● Example 4
date / time values which are dynamic
20 Built-in data-type, No Yes, includes ​RegEx and Macros
conditional-logic and
RegEx validations
21 Validate schema of all No Yes
elements in a JSON
 array in one step
22 Built-in JSON Schema Yes RegEx and Macros​ support is sufficient For details on how Karate’s
and XML Schema (and far simpler) for most use cases. That approach is simpler and more
validation support said, users can ​easily add a Java lib​ via intuitive than JSON (or XML)
Karate’s Java interop - if needed. Schema see ​this link​.
23 Native support for No Yes No need to use double-quotes
expressing JSON or or “escape” characters.
XML in test-scripts "{ \"name\": \"Billie\" }" { name: 'Billie' }

"<cat name=\"Billie\"></cat>" <cat name="Billie"></cat>

You can also read from files
and re-use.
24 Example – JSON @Test ​public void Scenario​: ​lotto resource returns 200 Matching built-in, and more
lotto_resource_returns_200_with_expecte with expected id and winners
assertions d_id_and_winners​() {
readable syntax. Note the
Given​ path ‘​lotto​’, ​5 simpler way to specify path
​when​(). When​ method get parameters without
get("​/​lotto​/{id}​", ​5​). Then​ status ​200 placeholders.
​then​(). And​ match ​$.lotto.lottoId​ == ​5
statusCode(​200​). And​ match ​$.lotto.winners[*].winnerId
body("​lotto.lottoId​", equalTo(​5​), contains only ​[23, 54] For REST-assured, IDE
"​lotto.winners.winnerId​", formatting is a known
containsOnly(​23​, ​54​));
}
challenge​.
25 Example - GET with given​(). Given​ ​param​ ​key1​ = ‘​value1​’ Karate is a ​true DSL​. No syntax
param("​key1​", "​value1​"). And​ param ​key2​ = ‘​value2​’
params param("​key2​", "​value2​"). And​ path ‘​somewhere​’
“noise”, no unnecessary
when​(). When​ method get symbols or punctuation. No
get("​/somewhere​"). Then​ match response contains ‘​OK​’ need to worry about indenting
then​().
a giant “one liner” of Java
body(containsString("​OK​"));
code.
26 Extracting multiple Convoluted. Easy. You can even use JsonPath to Some of the ​quirks​ of the
data-elements for extract JSON chunks or arrays and save REST-assured JsonPath
reuse in subsequent The Fluent Interface which is supposed to them to variables for use in later steps. implementation get in the way
HTTP calls be the main highlight of REST-Assured as well.
actually ​gets in the way​ here. More For XML, XPath does the same.
examples​.
27 Can update a given No​. Yes. There are actually multiple ways to You can even modify a
JSON or XML using a update payloads: a) by path b) using response and re-use it ‘as-is’
path expression Especially for data-driven tests, updating embedded expressions and c) via a as the next request.
nested JSON is ​near impossible​. built-in string replacement keyword.
28 Data Driven Testing No (have to use TestNG or equivalent) Yes. Can even use dynamic JSON or even
CSV as a data-source.
REST-Assured Example Karate Example
29 SOAP support No Yes Plus, Karate’s XML support is
far more ​flexible and easier to
use​.
30 HTTPS / SSL without Although there is “​relaxed​” HTTPS, a Yes
certificates certificate is ​needed​ in some cases
31 Built-in support for No Yes. In REST-assured, you have to
switching use something like a
environment config Also config is somewhat ​convoluted​ in Adding a new variable to a test is just dependency-injection
REST-Assured one step: edit ​karate-config.js framework (or roll your own)
to read properties files.
32 File Upload / Partial / Buggy Yes
Multipart Support Libraries​ | ​Content-Type​ | ​Dependencies​ |
‘multipart/related’ ​not supported​ |
questions​ on ‘multipart/mixed’
33 URL encoded HTML Yes Yes
Form data
34 Cookies Yes Yes
35 Auth Schemes out of Yes No (but ​easily pluggable​ via re-usable
the box scripts or JavaScript without needing to
write Java code)
36 Custom Auth Java code (needs compilation). Existing Unified plug-in system via JavaScript (no
mechanism is ​not extensible​. compilation needed)
37 Parallel Execution of No. While ​some teams​ seem to have had Yes This is a ​critical​ requirement
Tests success running REST-assured in parallel, for HTTP integration tests
there are ​some cases​ in which Even if you run tests in parallel, reports which typically take a much
multi-threading is not supported. Also see and logs are collected per test and HTML longer time than unit tests.
this thread​. The creator has also confirmed reporting works as you would expect.
that “​REST Assured is not 100% safe to use
in multi-threaded scenarios​”.

38 Floating-point All numbers are ​converted to float​ and you Numeric assertions work just as you
precision shouldn’t forget to use floats (not the expect and even auto-conversion to Even this works:
default double) in assertions. BigDecimal happens if needed.
And ​$.odd.ck​ ==
get​("​/odd​") Given​ path ‘​odd​’ 12.2000000000000
.​then​().assertThat() When ​method get
.body("​odd.ck​", equalTo(​12.2f​)); Then ​$.odd​ contains { ​ck​: ​12.2​ }

39 Lines of Code Needed More. By nature, Java is verbose and Less. Another example of how Java
to express a test especially if you depend on POJO “gets in the way” - the
 representations of payloads - you need This particular comparison​ shows a contortions you need to do​ to
more Java code in place. dramatic difference, 431 lines of code handle JSON arrays in
reduced to 67 REST-assured.
40 Test Reports Built-in No, you have to use JUnit, TestNG or Karate has text and HTML reports out of Here is an example​ of the very
equivalent for test reporting. the box and you get the option of nice-looking reports you can
choosing from the Cucumber ecosystem get by using the
of 3rd party reports. cucumber-reporting​ library.
41 Test any Java servlet REST-assured has support for Karate v0.5.0 onwards has support for This is possible because of
or HTTP resource “out-of-container” testing of specifically testing ​any​ servlet by providing Karate’s pluggable abstraction
without a container Spring-MVC​ but your tests will be extension points for teams to write an of the HTTP Client. Refer to
“hard-coded” in this mode. adapter. the documentation​ for more
details.
There is no support for things like JAX-RS or The huge advantage of Karate’s
custom servlets or controllers - and for approach is that the same test-script can You will be able to quickly
these you have to deploy to an app-server. be re-used for http-integration tests implement a custom adapter
without​ changes. for any Java server-side stack
in a similar way.
42 Report includes HTTP No. Karate 0.6.0 onwards includes ​HTTP
request and response request and response logs​ in the JSON
logs in-line report output. If you use the ​print
keyword, the console output appears in
the report as well, which is great for
troubleshooting. All this works even
when tests are run in parallel.
43 Construct JSON or No. Karate’s ​set​ keyword was enhanced in
XML from scratch v0.6.0 to support a ‘builder’ approach
using just path using cucumber tables. This is best
expressions explained via ​some examples​.
44 Test Doubles or No. Karate 0.7.0 onwards has support for Karate is a superior alternative
Mocks built-in creating API mocks​ that can fully to Wiremock, here’s a
You have to use 3rd party frameworks such simulate stateful CRUD. Having both a comparison​.
as ​Wiremock client and server in the same unified
framework keeps things simple and you Also see this ​perf benchmark​.
can move fast.
45 Performance Testing No. Karate 0.8.0 onwards has support for This alone is reason to choose
re-using Karate tests as ​Gatling Karate and no other
Also see [37] performance tests​. open-source test-automation
framework has this option.
You can compose multiple Karate
feature files or “work-flows” into a single In 0.9.0 onwards you can test
performance-test and use Gatling to any Java code​, not just HTTP
define the load-model (ramp-up,
concurrent users, etc)
46 Websockets support No​. Karate 0.9.0 onwards has ​support for
websockets​ and even generic async /
await support (see below)
47 Async Support No, you have to use Java code or a library Karate 0.9.0 has built-in support via a
like ​Awaitility very ​elegant API​.
48 Retry Support No, you have to write custom Java logic. Karate 0.9.0 onwards has this built-in -
where you ​only specify the condition​ to
be polled for.
49 CSV file support No, you have to use Java code or a library. In Karate 0.9.0 you can read a CSV file Karate also has YAML file
and it will be auto-converted to JSON support out of the box.
50 Web-UI Automation No. In Karate 0.9.5 onwards ​Web-UI
automation​ is possible.
51 Distributed Testing No. In Karate 0.9.5 onwards, there is ​built-in
support​ to split a test-suite and
orchestrate the execution across
multiple cloud or hardware / cluster
nodes