Beruflich Dokumente
Kultur Dokumente
magazine
By and for the Java community
JUnit 5
Arrives!
14 20 25 36 43 SPECIAL
WHATS NEW RUNNING JUNIT 5 KENT BECK ON MUTATION ISSUE
IN JUNIT 5? TESTS FROM EXTENSIONS HIS EVOLVING TESTING
JUNIT 4 AND 5 VIEWS OF
TESTING NOVEMBER/DECEMBER 2016
ORACLE.COM/JAVAMAGAZINE
//table of contents /
20 25 36 43
PART 2: A DEEP DIVE INTERVIEW MUTATION TESTING:
USING JUNIT 5 INTO JUNIT 5S WITH KENT BECK AUTOMATE THE
By Mert alikan EXTENSION MODEL By Andrew Binstock SEARCH FOR
Integrating with build By Nicolai Parlog The parent of JUnit and IMPERFECT TESTS
tools and IDEs and The lowdown on how creator of TDD discusses By Henry Coles
running JUnit 5 tests JUnit runs tests and how programming and testing Locate incorrect and
14 with earlier versions it interacts with libraries
and frameworks
and how his views on
testing have evolved.
incomplete unit tests
with pitest.
PART 1: A FIRST
LOOK AT JUNIT 5
By Mert alikan
The long-awaited release of JUnit 5 is a
complete redesign with many useful additions.
COVER ART BY I-HUA CHEN
03 11 61 67
From the Editor JavaOne 2016 Community Participation Fix This
NetBeans Gets a New Life The news, the videos, and the Contribute to Java By Simon Roberts
or Does It? Dukes Choice Awards by Adopting a JSR Our latest code quiz
By Martijn Verburg 54
06 55
Letters to the Editor New to Java How to stand out in the community Java Proposals of Interest
Comments, questions, suggestions, Implementing Design by contributing to Java standards JSR 367: JSON Binding
and kudos Patterns with Lambdas 63 73
07 By Raoul-Gabriel Urma, Mario Fusco, Cloud User Groups
and Alan Mycroft Getting Started with Agile
Events Astute use of lambdas can greatly
Istanbul JUG
Upcoming Java conferences and events Java Development in the Cloud
reduce the complexity of implementing By Shay Shmeltzer 74
standard coding patterns. Avoid endless setup with Oracle Contact Us
Developer Cloud Services integrated Git, Have a comment? Suggestion?
build, deploy, code review, and project Want to submit an article proposal?
management tools. Heres how.
01
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
EDITORIAL PUBLISHING
Editor in Chief Publisher
Andrew Binstock Jennifer Hamilton +1.650.506.3794
Managing Editor Associate Publisher and Audience
13 Billion
Claire Breen Development Director
Copy Editors Karin Kinnear +1.650.506.1985
Karen Perkins, Jim Donahue Audience Development Manager
Technical Reviewer Jennifer Kurtz
Stephen Chin
ADVERTISING SALES
DESIGN Sales Director
Senior Creative Director
Francisco G Delgadillo
Design Director
Richard Merchn
Tom Cometa
Account Manager
Mark Makinney
Account Manager
Devices Run Java
Senior Designer Marcin Gamza
Arianna Pucherelli Advertising Sales Assistant
Designer Cindy Elhaj +1.626.396.9400 x 201
Jaime Ferrand Mailing-List Rentals
Senior Production Manager Contact your sales representative.
Sheila Brennan
Production Designer RESOURCES ATMs, Smartcards, POS Terminals, Blu-ray Players,
Kathy Cygnarowicz
Oracle Products
+1.800.367.8674 (US/Canada) Set Top Boxes, Multifunction Printers, PCs, Servers,
Oracle Services
+1.888.283.0591 (US)
Routers, Switches, Parking Meters, Smart Meters,
Lottery Systems, Airplane Systems, IoT Gateways,
ARTICLE SUBMISSION Programmable Logic Controllers, Optical Sensors,
If you are interested in submitting an article, please email the editors.
SUBSCRIPTION INFORMATION Wireless M2M Modules, Access Control Systems,
Subscriptions are complimentary for qualified individuals who complete the
subscription form. Medical Devices, Building Controls, Automobiles
MAGAZINE CUSTOMER SERVICE
java@halldata.com Phone +1.847.763.9635
PRIVACY
Oracle Publishing allows sharing of its mailing list with selected third parties. If you prefer
that your mailing address or email address not be included in this program, contact
Customer Service.
Copyright 2016, Oracle and/or its affiliates. All Rights Reserved. No part of this publication may be reprinted or otherwise
reproduced without permission from the editors. JAVA MAGAZINE IS PROVIDED ON AN AS IS BASIS. ORACLE EXPRESSLY
DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED. IN NO EVENT SHALL ORACLE BE LIABLE FOR ANY #1 Development Platform
DAMAGES OF ANY KIND ARISING FROM YOUR USE OF OR RELIANCE ON ANY INFORMATION PROVIDED HEREIN. Opinions
expressed by authors, editors, and intervieweeseven if they are Oracle employeesdo not necessarily reflect the views of Oracle.
The information is intended to outline our general product direction. It is intended for information purposes only, and may not
be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied
upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles
products remains at the sole discretion of Oracle. Oracle and Java are registered trademarks of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.
Java Magazine is published bimonthly and made available at no cost to qualified subscribers by
Oracle, 500 Oracle Parkway, MS OPL-3A, Redwood City, CA 94065-1600.
02
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//from the editor /
Java in
the Cloud
NetBeans Gets a New Lifeor Does It? Oracle Cloud delivers
The transition from Oracle to the Apache Software Foundation marks the high-performance and
beginning of an uncertain new era for the Java IDE. battle-tested platform
and infrastructure services
for the most demanding
Java apps.
A t JavaOne this year, the NetBeans commu-
nity announced that the project was moving
from its longtime home at Oracle to the Apache
from Oracle). What few readers might know is
that NetBeans was the irst of these products
beating Borlands JBuilder by a year. (JDeveloper, Oracle Cloud.
Software Foundation (ASF). In a history that which was based on JBuilder, was next, followed Built for modern app dev.
dates back some 20 years, this will be NetBeans years later by Eclipse and IntelliJ.) Built for you.
ifth new home, showing the products remark- NetBeans became a popular Java IDE because
able power of endurance. An important question of several features, most especially the lightness
is whether working under the aegis of the ASF of its use. While competing products had a long
will bring NetBeans new life and new aiciona- setup cycle for new projects and a comparatively
dos, or whether it signals the inal chapter of a heavy feel, NetBeans was great for coding
storied lifeline. on the ly and always felt light and responsive.
As many readers know, NetBeans is one of While it lacked some of its competitors code-
the four principal Java IDEs. The others are the management features, it was the irst to ofer a
Start here:
open source Eclipse from the Eclipse Foundation, built-in execution proiler and, if I recall correctly,
developer.oracle.com
IntelliJ IDEA from JetBrains (consisting of an open the only one to include a small-scale J2EE server,
source version and a higher-end closed source OC4J, to quickly test web projects locally. It was
version), and JDeveloper (a free, closed source IDE also the irst IDE to ofer a top-quality Swing- #developersrule
32 IOT AND
THE CLOUD for years. Dallas, TX
One huge indicator of this problem: The manual
ORACLE.COM/JAVAMAGAZINE
on Java language style (which should be the bible Back Issues Prior to 2015
SEPTEMBER/OCTOBER 2016
here) has not been updated since 1999. I was reading something on the web that mentioned
The article says, The convenience and beneits an article in an older version of Java Magazine. How-
of such strictures that ensure uniform syntax are ever, when I look on the magazine home page it does
widely recognized. I agree. So, when will Oracle not show any issues before 2015. Is there any way to
update the Java language style manual and ofer guid- access the back issues? Even if theyre just in PDF
ance to developers and organizations? format, that would be helpful.
Alan Zeichick Michael Szela
Phoenix, AZ Palatine, IL
Andrew Binstock responds: I contacted the Java team to Andrew Binstock responds: We recently made the 2015
get more detail on this. They told me that the document and 2016 back issues available for viewing on the web
youre referring to was not posted as an attempt to codify and downloading as PDF iles. We plan to make the previ-
the language and therefore be a regularly updated docu- ous issues of Java Magazine available soon, most likely as
ment. Rather, it was a set of internal coding guidelines downloadable PDF iles for subscribers. Well announce the
published in response to a community request. The team availability of each year in the letter that goes out to sub-
suggested that the most normative examples of Java style scribers with each new issue of the magazine. We grate-
are those used in Oracles Java Tutorials. fully appreciate your patience.
Most books available today on Java style were writ-
ten a long time ago, alas. A volume from 2014 thats a Contact Us
worthy reference is Java Coding Guidelines, by Fred Long We welcome your comments and suggestions. Write
and a group of security experts. It not so much deines a to us at javamag_us@oracle.com. For other ways to
style as provides coding guidelines for writing secure code. reach us, see the last page of this issue.
06
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//events /
ArchConf in English. Several tracks are
DECEMBER 1215, 2016 usually devoted speciically to
CLEARWATER, FLORIDA Java EE, and the conference
ArchConf is an educational can be attended online.
event for software archi-
tects, technical leaders, and DevNexus
senior developers presented FEBRUARY 2022
by the No Fluf Just Stuf soft- ATLANTA, GEORGIA
ware symposium. Among the DevNexus is devoted to con-
slated sessions are Modularity, necting developers from all
Microservices, and Modern over the world, providing
Architectural Paradigms, led afordable education, and pro-
by Kirk Knoernschild, author of moting open source values.
Prentice-Halls Java Application The 2017 conference will take
Architecture, and Cloud-Native place at the Georgia World
Application Architecture, Congress Center in down-
led by Matt Stine, author of town Atlanta. Presenters will
OReillys Migrating to Cloud- include Josh Long, author of
Native Application Architectures. OReillys upcoming Cloud
Native Java: Designing Resilient
DevConf.cz 2017 Systems with Spring Boot, Spring
Jfokus FEBRUARY 6, UNIVERSITY DAY; FEBRUARY 78, CONFERENCE JANUARY 2729 Cloud, and Cloud Foundry,
STOCKHOLM, SWEDEN BRNO, CZECH REPUBLIC and Venkat Subramaniam,
Jfokus is the largest annual Java developer conference in Sweden. DevConf.cz 2017 is a free author of Pragmatics
Conference topics include Java SE and Java EE, continuous delivery and three-day open-source Fedora Functional Programming in Java:
DevOps, IoT, cloud and big data, trends, and JVM languages. This year, Linux and JBoss community Harnessing the Power of Java 8
the irst day of the event will include a VM Tech Summit, which is an conference for Red Hat and Lambda Expressions.
open technical collaboration among language designers, compiler writ- community developers,
ers, tool builders, runtime engineers, and VM architects. The schedule DevOps professionals, testers, Voxxed Days Zrich
will be divided equally between traditional presentations of 45 minutes and documentation writers. FEBRUARY 23
and informal, facilitated deep-dive discussion groups among smaller, Set to be hosted at the Brno ZRICH, SWITZERLAND
self-selected participants. Space is limited, as this summit is organized University of Technology, Sharing the Devoxx philoso-
around a single classroom-style room to support direct communication all talks, presentations, and phy that content comes irst,
between participants. workshops will be conducted Voxxed Days events see both
PHOTOGRAPH BY CHAS B/FLICKR 07
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//events /
spread of knowledge and innova- gathering of embedded system
tion in the developer commu- developers is Securely Connecting
nity. Scheduled tracks this year the Embedded World. Topics
include Performance Mythbusting include IoT, connectivity, software
and Every Last Nanosecond: Low engineering, and security.
Latency Java.
Devoxx US
jDays MARCH 2123
MARCH 78 SAN JOSE, CALIFORNIA
GOTHENBURG, SWEDEN Devoxx US focuses on Java, web,
jDays brings together software mobile, and JVM languages. The
engineers around the world to conference includes more than
share their experiences in difer- 100 sessions in total, with tracks
ent areas such as Java, software devoted to server-side Java, archi-
engineering, IoT, digital trends, tecture and security, cloud and
testing, agile methodologies, containers, big data, IoT, and more.
and security.
JavaLand
ConFoo Montreal 2017 MARCH 2830
internationally renowned and and agile development. Presen- MARCH 810 BRHL, GERMANY
local speakers converge. Past pre- tations this year include Java MONTREAL, QUEBEC, CANADA This annual conference features
sentations have included Bringing Libraries You Cant Aford to ConFoo Montreal is a multi- more than 100 lectures on sub-
the Performance of Structs to Java Miss, 8 Akka Antipatterns technology conference for web jects such as core Java and JVM
(Sort Of) by Simon Ritter and Java Youd Better Be Aware Of, and developers that promises 155 languages, enterprise Java and
Security Architecture Demystiied Spring Framework 5: Reactive presentations by popular inter- cloud technologies, IoT, front-end
by Martin Toshev. Microservices on JDK 9. national speakers. Past ConFoo and mobile computing, and much
topics have included how to write more. Scheduled presentations
Topconf Linz 2017 QCon London 2017 better streams with Java 8 and an include Multiplexing and Server
FEBRUARY 28, WORKSHOPS MARCH 68, CONFERENCE introduction to Java 9. Push: HTTP/2 in Java 9, The Dark
MARCH 12, CONFERENCE MARCH 910, WORKSHOPS and Light Side of JavaFX, JDK 8
LINZ, AUSTRIA LONDON, ENGLAND Embedded World Lambdas: Cool Code that Doesnt
Topconf covers Java and JVM, For more than a decade, QCon MARCH 1416 Use Streams, Migrating to Java 9
DevOps, reactive architecture, London has empowered software NUREMBERG, GERMANY Modules, and Java EE 8: Java EE
innovative languages, UX/UI, development by facilitating the The theme for the 15th annual Security API.
PHOTOGRAPH BY CRALVAREZ/FLICKR 08
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//events /
OReilly Software Architecture J On The Beach JCrete the Java and JVM ecosystem,
Conference MAY 17 AND 20, WORKSHOPS JULY 1721 DevOps, and agile development
APRIL 23, TRAINING MAY 1819, TALKS KOLYMBARI, GREECE environments.
APRIL 35, TUTORIALS MALAGA, SPAIN This loosely structured uncon-
AND CONFERENCE JOTB is an international rendez- ference involves morning ses- JavaOne
NEW YORK, NEW YORK vous for developers interested in sions discussing all things Java, OCTOBER 15
This event promises four days of big data technologies. JVM and combined with afternoons spent SAN FRANCISCO, CALIFORNIA
in-depth professional training .NET technologies, embedded and socializing, touring, and enjoy- Whether you are a seasoned
that covers software architecture IoT development functional pro- ing the local scene. There is also a coder or a new Java programmer,
fundamentals; real-world case gramming, and data visualization JCrete4Kids component for intro- JavaOne is the ultimate source of
studies; and the latest trends in will all be discussed. Scheduled ducing youngsters to program- technical information and learn-
technologies, frameworks, and speakers include longtime Java ming and Java. Attendees often ing about Java. For ive days, Java
techniques. Past presentations Champion Martin Thompson and bring their families. developers gather from around
have included Introduction to Red Hat Director of Developer the world to talk about upcom-
Reactive Applications, Reactive Experience Edson Yanaga. berConf ing releases of Java SE, Java EE,
Streams, and Options for the JULY 1821 and JavaFX; JVM languages; new
JVM, as well as Microservice QCon New York DENVER, COLORADO development tools; insights into
Standardization. JUNE 2628, CONFERENCE berConf 2017 will be held at the recent trends in programming;
JUNE 2930, WORKSHOPS Westin Westminster in down- and tutorials on numerous related
Devoxx France NEW YORK, NEW YORK town Denver. Topics include Java and JVM topics.
APRIL 5, WORKSHOPS QCon is a practitioner-driven Java 8, microservice architectures,
APRIL 67, CONFERENCE conference for technical team Docker, cloud, security, Scala, Have an upcoming conference
PARIS, FRANCE leads, architects, engineering Groovy, Spring, Android, iOS, youd like to add to our listing?
Devoxx France presents keynotes directors, and project manag- NoSQL, and much more. Send us a link and a description of
from prestigious speakers, then ers who inluence innovation your event four months in advance
a cycle of eight mini conferences in their teams. Past speakers NFJS Boston at javamag_us@oracle.com.
every 50 minutes. You can build include Chris Richardson, author SEPTEMBER 29OCTOBER 1
your own calendar and follow the of POJOs in Action, and Frank BOSTON, MASSACHUSETTS
sessions as you wish. Founded by Greco, organizer of the largest Since 2001, the No Fluf Just Stuf
developers for developers, Devoxx Java user group in North America (NFJS) Software Symposium Tour
France covers topics ranging from (NYJavaSIG). has delivered more than 450
web security to cloud computing. events with more than 70,000
(No English page available.) attendees. This event in Boston
covers the latest trends within
09
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//javaone recap /
JavaOne 2016:
The News and the Videos
J avaOne, the annual meeting of Java developers, was
held this year as usual in San Francisco, California,
in mid-September. This conference has long been the
the IDE for the next two releases onlythat is, the
imminent 8.x release and the later version 9, which
will support Java 9. In response, the NetBeans commu-
largest congregation of Java programmers in the world. nity has applied to become a project under the aegis of
This year, more than 9,000 developers participated in the Apache Software Foundation. The ramiications of
hundreds of sessions, many of which dealt with new this are discussed in detail in the editorial in this issue
technologies such as the upcoming JDK 9. (page 3).
There is news to report that came out of the con- As usual, there were many brilliant presentations at
ference. Perhaps most important is Oracles announce- JavaOne. Some 85 of them were videotaped and are now
ment of the forthcoming Java EE 8, which will be fol- posted on YouTube.
lowed by Java EE 9. Oracle Group Vice President Anil An annual ixture of JavaOne is the Dukes Choice
Gaur gave some details on version 8, which is expected Awards, which recognize particularly meritorious Java
to ship toward the end of 2017. It will include enhanced projects and Java tools. These awards are chosen by a
security (management of secrets and support for OAuth panel of community leaders and Oracle technical staf.
and OpenID), greater support for reactive program- This year, the judges selected the following recipients
ming, a uniied event model, HTTP/2, JSON-B, as well of interest to developers: PrimeFaces (theres a pattern
as a large set of other technologies. In addition, it will herelast year the judges chose OmniFaces); HeapStats,
be optimized for serverless contexts and for containers. a lightweight monitoring tool for Java heap and GC
Java EE 9 is expected to ship a year later, in late 2018. status with a GUI front end for analyzing the data it
(A video of Gaurs presentation on Java EE 8 is available generates; and Strata, an open source package used for
on YouTube.) inancial analytics and market risk estimation.
Another event of importance that happened during The next annual JavaOne conference will be held
JavaOne, but not strictly speaking at the conference, October 15, 2017, in San Francisco. For a listing of other
was the changing fate of the NetBeans IDE. Oracle conferences and events, see the Events section (page 7)
announced that it would be the primary sponsor of in this issue.
11
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
SPECIAL ISSUE
J
Unit is the most widely used testing tool in Java. Survey after ment the design that tools use to drive or interact with JUnit.
survey shows this. Its so pervasive that other testing tools are But this is a special issue, so theres plenty more here. On page 36,
frequently built on top of JUnit (Spock, for example, as well as we interview Kent Beck, the original parent of JUnit as well as the father
most of the behavior-driven development frameworks), rather of extreme programmingthe core set of practices that form the basis of
than trying to duplicate its capabilities. JUnits speed, ease of most modern software development. As youll see, Becks view on test-
use, and universality make it the Java developers universal tool. ing has evolved from being deeply rooted in test-irst development to one
Version 5 is a fundamental rewrite and rearchitecture of JUnit. The that balances the beneits of tests with the costs they impose. He explains
new features of this release are this more nuanced view in detail,
summarized in our irst article which is sure to give pause to die-
(page 14), which gives a concise hard TDD fans.
overview of the reinements. The Finally, for developers who rely
second article (page 20) shows how on unit testing as a backstop for
to include JUnit 5 in your toolchain mission-critical code, we explore
and, especially, how to run tests mutation testing (page 43), which
for versions 4 and 5 in the same is heavy-duty, automated testing
testing run. that searches for gaps in unit tests.
Now that you see the beneits It takes the unit tests and modi-
of this release, we take you for a ies them slightly to see if tested-
deep dive into the architecture for conditions when changed or
(page 25). This article is ideal for removed from the test cause the
developers who want to extract test to fail or throw an exception.
the most capability from the new This can identify duplicate tests,
version, rather than just stay at a incomplete tests, and those that
basic assertion level. Its also an dont actually test what you expect.
excellent introduction to JUnits May the green bar be good
extension points, which imple- to you!
ART BY I-HUA CHEN
13
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
MERT ALISKAN
J Unit, the widely used Java unit testing framework, has
just seen the irst alpha release of version 5 after 10 years
on version 4. JUnit 5 consists of a revamped codebase with a
Anatomy of a JUnit 5 Test
Lets look at some JUnit 5 tests, starting with the simple JUnit
test shown in Listing 1.
modular architecture, a new set of annotations, an extensible
model for third-party library integration, and the ability to Listing 1.
use lambda expressions in assertions. import org.junit.jupiter.api.*;
The predecessor of JUnit 5 was the JUnit Lambda proj-
ect, which sowed the irst ideas for the next generation of class SimpleTest {
unit testing and was crowd-funded until October 2015 on
Indiegogo, where it received more than twice the target @Test
amount in contributions. void simpleTestIsPassing() {
Through the years, JUnit has captured the essence of org.junit.jupiter.api.Assertions
what a unit testing framework should be. However, its core .assertTrue(true);
mostly stayed intact, which made it diicult for it to evolve. }
This new version is a complete rewrite of the whole product }
that aims to provide a suicient and stable API for running
and reporting tests. Implementing unit tests with JUnit 5 For a simple JUnit 5 test class, such as the one shown here,
requires Java 8 at a minimum, but it can run tests on code there is almost no diference to be seen at irst glance when
written for earlier versions of Java. compared with a JUnit 4 test class. The main diference is
In Part 1 of this article, I describe the principal new fea- that there is no need to have test classes and methods
tures of JUnit 5, illustrating them with detailed examples. deined with the public modiier. Also, the @Test annota-
The JUnit team is planning to ship the inal version of the tionalong with the rest of the annotationshas moved to
framework by the end of 2016. Milestone 2 is one of the last a new package named org.junit.jupiter.api, which needs
steps before JUnit 5 oicially ships. This will surely be one of to be imported.
the most consequential releases ever in the Java ecosystem.
In Part 2 (page 20), I explain how to use and conigure JUnit 5 Capitalizing on the Power of Annotations
with your existing tools and how to run tests from JUnit 4 and JUnit 5 ofers a revised set of annotations, which, in my view,
JUnit 5 together. provide essential features for implementing tests. The anno-
14
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
tations can be declared individually, or they can be composed ods, or both. Tagging provides a way of iltering tests for
to create custom annotations. In the following section, I execution. This approach is analogous to JUnit 4s Categories.
describe each annotation and give details with examples. Listing 4 shows a sample test class that uses tags.
@DisplayName. Its now possible to display a name for a test
class or its methods by using the @DisplayName annotation. Listing 4.
As shown in Listing 2, the description can contain spaces and @Tag("marvelous-test")
special characters. It can even contain emojis, such as J. @Tags({@Tag("fantastic-test"), @Tag("awesome-test")})
class TagTest {
Listing 2.
@DisplayName("This is my awesome test class ") @Test
class SimpleNamedTest { void normalTest() {
}
@DisplayName("This is my lonely test method")
@Test @Test
void simpleTestIsPassing() { @Tag("fast-test")
assertTrue(true); void fastTest() {
} }
} }
@Disabled. The @Disabled annotation is analogous to the You can ilter tests for execu- If an assumption
@Ignore annotation of JUnit 4, and it can be used to disable tion or exclusion by providing tag fails, it does not
the whole test class or one of its methods from execution. The names to the test runners. The
reason for disabling the test can be added as a description to way to run ConsoleLauncher is
mean the code is
the annotation, as shown in Listing 3. described in detail in Part 2 of this broken, but only that
article. With ConsoleLauncher, the test provides no
Listing 3. you can use the t parameter for
class DisabledTest { providing required tag names or
useful information.
the T parameter for excluding The default JUnit
@Test tag names. runner ignores such
@Disabled("test is skipped") @BeforeAll, @BeforeEach,
void skippedTest() { @AfterEach, and @AfterAll. The
failing tests. This
fail("feature not implemented yet"); behavior of these annotations is approach enables
} exactly the same as the behavior other tests in the
} of JUnit 4s @BeforeClass, @Before,
@After, and @AfterClass, respec-
series to be executed.
@Tags and @Tag. Its possible to tag test classes, their meth- tively. The method annotated with
15
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
@BeforeEach will be executed before each @Test method, and NoSuchElementException.class,
the method annotated with @AfterEach will be executed after items::remove);
each @Test method. The methods annotated with @BeforeAll }
and @AfterAll will be executed before and after the execution }
of all @Test methods. These four annotations are applied to
the @Test methods of the class in which they reside, and they @Nested
will also be applied to the class hierarchy if any exists. (Test class WhenWithOneElement {
hierarchies are discussed next.) The methods annotated with @Test
@BeforeAll and @AfterAll need to be deined as static. void addingOneElementShouldIncreaseSize() {
@Nested test hierarchies. JUnit 5 supports creating hierarchies items.add("Item");
of test classes by nesting them inside each other. This option assertEquals(items.size(), 1);
enables you to group tests logically and have them under the }
same parent, which facilitates applying the same initializa- }
tion methods for each test. Listing 5 shows an example of }
using test hierarchies.
Assertions and Assumptions
Listing 5. The org.junit.jupiter.api.Assertions class of JUnit 5
class NestedTest { contains static assertion methodssuch as assertEquals,
assertTrue, assertNull, and assertSameand their corre-
private Queue<String> items; sponding negative versions for handling the conditions in
test methods. JUnit 5 leverages the use of lambda expressions
@BeforeEach with these assertion methods by providing overloaded ver-
void setup() { sions that take an instance of java.util.function.Supplier.
items = new LinkedList<>(); This enables the evaluation of the assertion message lazily,
} meaning that potentially complex calculations are delayed
until a failed assertion. Listing 6 shows using a lambda
@Test expression in an assertion.
void isEmpty() {
assertTrue(items.isEmpty()); Listing 6.
} class AssertionsTest {
@Nested @Test
class WhenEmpty { void assertionShouldBeTrue() {
@Test assertEquals(2 == 2, true);
public void removeShouldThrowException() { }
expectThrows(
16
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
@Test ing them as an attribute to the @Test annotation. This is an
void assertionShouldBeTrueWithLambda() { enhancement compared with previous versions that required
assertEquals(3 == 2, true, the use of try-catch blocks for handling exceptions. JUnit 5
() -> 3 not equal to 2!); introduces the usage of lambda expressions for deining the
} exception inside the assertion statement. Listing 8 shows the
} placement of the exception directly into the assertion.
@Test Listing 9.
void groupedAssertionsAreValid() { class Exceptions2Test {
assertAll(
() -> assertTrue(true), @Test
() -> assertFalse(false) void expectingArithmeticException() {
); StringIndexOutOfBoundsException exception =
} expectThrows(
} StringIndexOutOfBoundsException.class,
() -> "JUnit5 Rocks!".substring(-1));
Expecting the Unexpected
JUnit 4 provides a way to handle exceptions by declar- assertEquals(exception.getMessage(),
17
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
"String index out of range: -1"); instance of Collection, Stream, Iterable, or an Iterator of
} type DynamicTest. The DynamicTest class denotes a test case
} that will be generated at runtime. Actually, its a wrapper
class that contains a name and an executable. That execut-
Dynamic Testing able refers to the test code execution block. The dynamicTest
With a new dynamic testing feature of JUnit 5, its now pos- static method deinition resides under the DynamicTest class,
sible to create tests at runtime. This was not possible prior and its objective is to create an instance of DynamicTest by
to version 5 because all testing code needed to be deined at retrieving a name and an instance of Executable, which con-
compile time. An example of dynamic test creation is shown sists of lambda expressions (as shown in Listing 10) that use
in Listing 10. two assertions.
The lifecycle of a dynamic test is diferent from that of a
Listing 10. standard @Test annotated method. This means that lifecycle
class DynamicTestingTest { callback methods, such as @BeforeEach and @AfterEach, are
not executed for dynamic tests.
@TestFactory
List<DynamicTest> Parameterized Test Methods
createDynamicTestsReturnAsCollection() { With the help of dynamic testing in JUnit 5, its possible to
return Arrays.asList( execute the same test with diferent data sets. This was also
dynamicTest("A dynamic test", possible in JUnit 4 by employing the Parameterized runner
() -> assertTrue(true)), and by deining data with the @Parameterized.Parameters
dynamicTest("Another dynamic test", annotation. But that approach has a limitation: it runs all test
() -> assertEquals(6, 3 * 2)) methods annotated with @Test for every parameter again and
); again, leading to needless executions. Creating dynamic tests
} for each data item could lead to better encapsulation that is
} local to the test method. I demonstrate this in Listing 11.
Get Java
The dynamicSquareRootTest method annotated with
@TestFactory is not a test case, but it creates a Stream
instance of the DynamicTest class, which will contain the
various test implementations. For each tuple element of
the stream, I execute a lambda expression that maps to an
executable test case, and that test case does an assertion
that tests whether the irst element of the tuple equals
the square root of the second element of the tuple. Isnt
that elegant?
Certified
Conclusion
The JUnit team has succeeded in ofering a new, redesigned Oracle University
version of JUnit that addresses nearly all the limitations of
previous versions. Note that the JUnit 5 API is still subject
to change; the team is annotating the public types with the
@API annotation and assigning values such as Experimental,
Maintained, and Stable. Upgrade & Save 35%*
Give JUnit 5 a spin, and keep your green bar always on!
learn more Save 35% when you upgrade or advance your existing Java certification. Offer expires December 31, 2016. Click for further details, terms, and conditions.
MERT ALISKAN I n Part 1 of this article (page 14), I wrote about the features
coming in JUnit 5. In this article, I provide more details on
the framework and its integration with build tools such as
tests from the command line and printing execution results
back to the console. The junit-platform-surefire-provider
module contains the JUnitPlatformProvider class, which
Maven and Gradle. integrates with the Sureire plugin to run JUnit 5 tests via
All examples in this article are based on JUnit version Maven. (Sureire is the Maven plugin that runs JUnit tests
5.0.0-M2, which can be found on the project home page. during the test cycle.) In addition, the junit-platform-
Architectural Overview
Lets start with the packaging structure of JUnit 5, which was JUnit 4 JUnit 5
revised after its alpha release. JUnit 5 now consists of three based test based test
main packages: Platform, Jupiter, and Vintage. The packaging
structure and modules are illustrated in Figure 1, which is cur- junit 4.12 junit-jupiter-api
VINTAGE
JUPITER
rent as of the M2 release.
JUnit Platform is the package that provides the foun- junit-vintage-engine junit-jupiter-engine
dation for the Vintage and Jupiter packages. It contains the
junit-platform-engine module, which ofers a public API for
integration of third-party testing frameworks such as Specsy,
which is a behavior-driven development (or BDD)-style unit-
junit-platform-engine
level testing framework for JVM languages. The launching API
Backward Compatibility
with Vintage Mode
With the help of the JUnit
Platform class, which is an
implementation of JUnit 4s
Runner, its possible to execute
JUnit Jupiter-based test classes
on JUnit 4. The junit-
platform-runner module
contains the needed
JUnitPlatform class and it
Figure 2. Output from IntelliJ for nested tests should be deined with the
23
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
Jupiter engine dependency, as shown in Listing 9, in Maven. }
}
Listing 9.
<dependency> Conclusion
<groupId>org.junit.jupiter</groupId> The JUnit team did a very good job with the latest release of
<artifactId>junit-jupiter-engine</artifactId> version 5, and the new packaging structure shows that the
<version>5.0.0-M2</version> framework has been revamped to provide a foundation for
<scope>test</scope> many future releases. JUnit 5 addresses nearly all the limi-
</dependency> tations of the previous version and provides better support
<dependency> via integration points for build tools, IDEs, and third-party
<groupId>org.junit.platform</groupId> testing frameworks. By leveraging the use of lambdas and
<artifactId>junit-platform-runner</artifactId> with the help of new implementations such as the extension
<version>1.0.0-M2</version> model, I believe JUnit will continue to be the most popular
<scope>test</scope> Java framework. </article>
</dependency>
Mert aliskan (@mertcal) is a Java Champion and coauthor
A sample test case implementation is given in Listing 10. As of PrimeFaces Cookbook (Packt Publishing, irst edition, 2013;
seen in the import statements, the test case is solely imple- second edition, 2015) and Beginning Spring (Wiley Publications,
mented with JUnit 5, and the deined runner makes it pos- 2015). He is the founder of Ankara JUG, the most active Java user
sible to execute the test on JUnit 4based platforms such as group in Turkey.
Eclipse Neon.
Listing 10.
import org.junit.jupiter.api.Test; learn more
import org.junit.platform.runner.JUnitPlatform;
import org.junit.runner.RunWith; In-depth blog on the JUnit architecture
Using JUnit 5 in IntelliJ IDEA
import static
JUnit team on Twitter
org.junit.jupiter.api.Assertions.assertTrue;
separates JUnit the tool from JUnit the platform and a new @BeforeEach
extension model that does away with key limitations of the static void setUp() { /* <5.> */ }
previous architecture.
In this article, I examine the extension model, which is @Test
what third-party libraries and frameworks use to integrate // @Disabled <3.>
with or extend JUnit. This topic will be of primary interest to void testMethod(String parameter /* <6.> */)
writers of those tools and libraries, as well as to developers { /* <7. then 8.> */ }
who want intimate knowledge of how JUnit works. To follow
along, you will need to have a good working knowledge of @AfterEach
JUnit 4. static void tearDown() { /* <9.> */ }
I should add that application developers who spend the
time to understand the extension model will be in a position @AfterAll
to reduce boilerplate and improve the readability and main- static void tearDownOnce() { /* <10.> */ }
tainability of their tests.
}
Test Lifecycle
Extensions hook into the test lifecycle, so lets look at that Here are the steps in the lifecycle (the numbers refer to the
irst. Lets take the following test as an example: comments above):
1. Check whether the tests in the test class (called the
// @Disabled <1.> container by JUnit) should be executed.
class LifecycleTest { 2. An instance of the container is created.
3. Check whether the particular test should be executed.
LifecycleTest() { /* <2.> */ } (From a lifecycle point of view, this step occurs here.
25
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
However, from a coding point of view, this is generally AfterTestExecutionCallback are new. They are more of a
speciied in the location marked in the preceding code.) technical requirement in case an extension needs to run as
4. The @BeforeAll (formerly @BeforeClass) methods are closely to the test as possible, for example, to benchmark
called if this is the irst test from the container. a test.
5. The @BeforeEach (formerly @Before) methods are called. What exactly is an extension, and how does it interact
6. Test method parameters are resolved. (Note that test with extension points? For each of the extension points, there
methods can now have parameters!) exists a Java interface that has the same name. The interfaces
7. The test is executed. are quite simple and usually have one, sometimes two, meth-
8. Possible exceptions are handled. ods. At each point, JUnit gathers a lot of context information
9. The @AfterEach (formerly @After) methods are called. (Ill get to that shortly), accesses a list of registered exten-
10. The @AfterAll (formerly @AfterClass) methods are sions that implement the corresponding interface, calls the
called if this is the last test from the container. methods, and changes the tests behavior according to what is
Now lets see how to interact with that lifecycle. returned by the methods.
This is a custom annotation that a project might create void publishReportEntry(Map<String, String> map);
to fulill common requirements for integration tests. In
this case, all such tests are benchmarked and tagged as Store getStore();
integration, which allows them to be iltered and, most Store getStore(Namespace namespace);
importantly, annotated with @Test, which allows you to use
@IntegrationTest instead of @Test: JUnit creates a tree of test nodes, and each node produces
these contexts. Because the nodes have parents (for example,
class ServerTest { the node corresponding to a test class is parent to the nodes
corresponding to the methods it declares), they let their
@IntegrationTest extension context reference their parents context.
29
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
To enable you to identify and ilter containers and tests, it to say that it is a way to log
these items have IDs, more-human-readable display names, It is straightforward to run messages to diferent output
and tags, which are accessed using the context methods. Very old JUnit 4 tests and new sinks, such as the console
importantly, the context gives access to the class or method or XML reports. The method
it was created for. This enables extensions to use relection,
JUnit 5 tests side by side. publishReportEntry enables
which can, for example, access a tests annotations or a This means it is not an extension to interact with
classs ields. Lets see this in action by slightly enriching the necessary to migrate the report. Finally, there is a
benchmark extension to include the tests display name in data store that must be used to
the logged message:
individual tests. persist an extensions state. Ill
discuss this shortly.
@Override As Ive mentioned, JUnit
public void afterTestExecution( is in charge of identifying and applying extensions, which
TestExtensionContext context) { implies that it is also managing instances of the extensions.
long elapsedTime = How does it do that? If you are going to assign information you
System.currentTimeMillis() - launchTime; gathered during a run to ields, as I did in BenchmarkExtension,
System.out.printf("Test '%s' took %d ms.%n", you need to know the extensions scope and lifetime.
context.getDisplayName(), elapsedTime); As it turns out, thats an intentionally unspeciied imple-
} mentation detail. Deining a lifecycle for extension instances
and tracking them during a running test suite is at best both-
And you can go even further. Instead of crudely printing to ersome and at worst a threat to maintainability. So all bets
the console, you can use JUnits report infrastructure by call- are of! JUnit makes no guarantees whatsoever regarding the
ing publishReportEntry: lifecycle of extension instances. Hence, they need to be state-
less and should store any information on a data structure
@Override provided by JUnit for that speciic purpose: the store.
public void afterTestExecution( The store. A store is a namespaced, hierarchical, key-value
TestExtensionContext context) { data structure. Lets look at each of these properties in turn.
long elapsedTime = To access the store via the extension context, a name-
System.currentTimeMillis() - launchTime; space must be provided. The context returns a store that
String message = manages entries exclusively for that namespace. This is done
String.format("Test '%s' took %d ms.", to prevent collisions between diferent extensions operating
context.getDisplayName(), elapsedTime); on the same node, which could lead to accidental sharing
context.publishReportEntry( and mutation of state. (Interestingly enough, the access via
createMapWithPair("Benchmark", message)); namespaces can be used to intentionally access another
} extensions state, allowing communication and, hence, inter-
action between extensions, which could lead to interesting
I wont discuss JUnits reporting facility in depth, but suice cross-library features.)
30
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
Stores are hierarchical because a store is created for each private static final String LAUNCH_TIME_KEY =
extension context, which means there is one store per node in "LaunchTime";
the test tree. Each test container or test method has its own
store. In much the same way as nodes inherit extensions, Reading and writing are two simple methods:
stores inherit state. To be more precise, when a node creates
a store, the node hands the store a reference to its parents private static void storeNowAsLaunchTime(
store. Thus, for example, the store belonging to a test method ExtensionContext context) {
holds a reference to the store belonging to the test class that context.getStore(NAMESPACE)
contains the method. Upon queries (but not edits), a store .put(LAUNCH_TIME_KEY, currentTimeMillis());
irst checks its own content before delegating to its parent }
store. This makes a stores state readable to all child stores.
Regarding having a key-value data structure, a store is a private static long loadLaunchTime(
simpliied map in which keys and values can be of any type. ExtensionContext context) {
Here are the most essential methods: return context.getStore(NAMESPACE)
.get(LAUNCH_TIME_KEY, long.class);
interface Store { }
void put(Object key, Object value);
<V> V get(Object key, Class<V> requiredType); With these methods, I replace the access to the ield
<V> V remove(Object key, Class<V> requiredType); launchTime, which can subsequently be removed.
} The methods executed before and after each test now
look as follows:
The methods get and remove take a type token to prevent
clients from littering their code with casts. @Override
There is no magic there; the store simply does the casts public void beforeTestExecution(
internally. Overloaded methods without type tokens exist TestExtensionContext context) {
as well. storeNowAsLaunchTime(context);
Stateless benchmarks. To make the benchmark extension }
stateless, I need a couple of things:
A namespace for the extension to access a store @Override
A key for the launch time public void afterTestExecution(
The ability to write to and read from the store instead of TestExtensionContext context) {
from a ield long launchTime = loadLaunchTime(context);
For the irst two, I declare two constants: long runtime = currentTimeMillis() - launchTime;
print(context.getDisplayName(), runtime);
private static final Namespace NAMESPACE = }
Namespace.create("org", "codefx", "Benchmark");
31
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
As you can see, the new meth- @org.junit.jupiter.api.Test
ods use the store instead of the During the test public @interface Test { }
ield to persist and access the lifecycle, JUnit
extensions state. This is pretty straightforward: I just declare the annotation
pauses at each and meta-annotate it with JUnit 5s @Test, so JUnit will iden-
Retrofitting @Test extension point, tify annotated methods as test methods and run them.
Lets look at a new example that searches for all Expecting exceptions. To manage expected exceptions, I irst
leverages much of the material Ive need a way for the user to declare them. For this, I extend
just covered.
extensions that my annotation with code that is heavily inspired by JUnit 4s
Lets say I want to move from apply to the current implementation of this feature:
JUnit 4 to JUnit 5. First of all, thanks test node, gathers
to the design of the new architec- public @interface Test {
ture, it is straightforward to run
context information,
old and new tests side by side. This and calls extensions class None extends Throwable {
means it is not necessary to migrate in outside-in order. private static final long
individual tests, which makes what serialVersionUID = 1L;
follows a little moot but no less fun. private None() { }
I want to replace JUnit 4s @Test }
annotation with a new version that makes the annotated
method a JUnit 5 test. I could pick JUnit 5s @Test, and this Class<? extends Throwable>
works in most cases: a simple search-and-replace on the expected() default None.class;
import would do the trick. (Note: This is just a thought exper-
iment, not an actual recommendation.) }
But JUnit 4s optional arguments expected (to fail tests
when a particular exception is not thrown) and timeout (to Now, a user can use expected to specify which exception to
fail tests that run too long) are not supported in JUnit 5s expect. It defaults to None.
annotation. JUnit 5 provides these features via assertThrows The extension itself will be found in a class called
and the upcoming assertTimeout. But Im looking for a way ExpectedExceptionExtension, which is shown below. To reg-
that requires no manual intervention, which precludes updat- ister it with JUnit, I annotate @Test with @ExtendWith(Expected
ing the tests to the new API. ExceptionExtension.class).
So why not create my own @Test that JUnit 5 will recog- Next, I need to actually implement the desired behavior.
nize and run and that implements the desired functionality? Here is a short description of how I can do it:
First things irst. Ill declare a new @Test annotation: 1. If a test throws an exception, check whether it is the
expected exception. If it is, store the fact that the
@Target(METHOD) expected exception was thrown. Otherwise, store that it
@Retention(RetentionPolicy.RUNTIME) was the wrong one and rethrow the exception (because
32
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
the extension is not in charge of that exception).
TestExtensionContext context, Throwable throwable)
2. After the test is executed, check whether the expected
throws Throwable {
exception was thrown. If so, do nothing because every-
boolean throwableMatchesExpectedException =
thing went as planned; otherwise, fail the test.
expectedException(context)
For this logic, I need to interact with two extension .filter(expected ->
points, TestExecutionExceptionHandler and AfterTest expected.isInstance(throwable))
ExecutionCallback, so I implement the corresponding .isPresent();
interfaces: if (throwableMatchesExpectedException) {
storeExceptionStatus( context,
public class ExpectedExceptionExtension EXCEPTION.WAS_THROWN_AS_EXPECTED);
implements TestExecutionExceptionHandler, } else {
AfterTestExecutionCallback { storeExceptionStatus(context,
EXCEPTION.WAS_THROWN_NOT_AS_EXPECTED);
@Override throw throwable;
public void handleTestExecutionException( }
TestExtensionContext context, }
Throwable throwable)
throws Throwable { } Note that by not rethrowing the exception, I inform JUnit that
I processed it and that all is in order. Accordingly, JUnit will
@Override neither call additional exception handlers with it nor fail the
public void afterTestExecution( test. So far so good.
TestExtensionContext context) { } Now, after the test, I need to check what happened so I
} can react accordingly. Another helper, loadExceptionStatus,
will retrieve the state and do me a further small favor: when
Lets start with step 1 and check whether the exception is no exception was thrown, the extension point I implemented
the one expected. For this, I use a small utility function, above will not have been called, which means no EXCEPTION
expectedException, which accesses the @Test annotation, instance was placed into the store. In this case, loadException
extracts the expected exception class, and returns it in an Status will return EXCEPTION.WAS_NOT_THROWN. Here is the
Optional (because maybe no exception was expected). implementation:
To capture the observed behavior, I create an enum, EXCEPTION,
and to persist the observation in the store, I write store @Override
ExceptionStatus. With these helpers in place, I can imple- public void afterTestExecution(
ment the irst extension point: TestExtensionContext context) {
switch(loadExceptionStatus(context)) {
@Override case WAS_NOT_THROWN:
public void handleTestExecutionException( expectedException(context)
33
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
.map(expected -> new IllegalStateException( is that there can be no extensions that run tests on speciic
"Expected exception " + expected + threads, such as the event dispatch thread for Swing tests
" was not thrown.")) or the application thread for JavaFX tests. The JUnit team is
.ifPresent(ex -> { throw ex; }); well aware of this limitation. Lets hope they address it soon.
You could implement an alternative feature that mea-
case WAS_THROWN_AS_EXPECTED: sures how long a test ran, which implies that it must have
// the exception was thrown as expected, inished, and fail it if it was above the threshold. With all I
// so there is nothing to do discussed so far, this should be fairly straightforward.
34
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
ANDREW BINSTOCK Binstock: I understand you work at Facebook these days. What changes there? If I were to ask how your review of program-
is it that you do there? ming was informed by the concerns of scaling, what would
Beck: I am focused on engineer education. My oicial title is you say is diferent?
technical coach, and that means what I do most days is pair Beck: Its a great question because its really hard to boil it
program and talk with engineers. down, so I can give you some speciics. Logging is far more
Binstock: Are these typically seasoned engineers or those just important. Performance, in some cases, is far more impor-
recently entering the ield? tant. A tiny little performance regression can bring the entire
Beck: All sorts. What I ind if I coach a bunch of engineers site down. Because were trying to operate very eiciently
at a given level, Ill start spotting patterns among whatever in terms of capital and also in terms of CPUs and bandwidth
bottleneck theyre hitting, and frankly, and everything, theres very little head-
I get bored telling the same stories and room sometimes. So, for certain teams
addressing the same issues. So Ill write performance, theres a lot to lose, as well
a course that addresses those issues. as a little bit to gain, and thats, I think,
We have an organization thats very, unusual. Logging is all about being able
very good at cranking lots of engineers to debug after something horrible goes
through the course. So we have courses wrong. In classic extreme programming
for new college graduates; we have a style, you arent going to need it (YAGNI),
course for people making the transition so you dont write it. Well, here you are
to technical leadership; we have a course going to need it, so you do write it. Even
for technical leaders hired in from the if you dont end up ever needing it, you
outside, because Facebook culture is very, still need it.
very diferent, and if you are used to lead- Binstock: I see.
ing by giving commands that other people Beck: You need the option of being able to
obey, thats not going to work. post-mortem a service, and that options
Binstock: When youre working in a place worth a lot in a way that I just had never
like Facebook, youre probably seeing a Kent Beck, inventor of extreme programming seen before.
diferent kind of scaling dimension than and cocreator of JUnit. His work led to the Binstock: How about when you commit
PHOTOGRAPH BY BOB ADLER/
GETTY IMAGES most developers encounter. So what popularity of developer-based testing. code to the main trunk? I would imagine
36
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
that the amount of test- because its going to be deployed on hundreds of thousands
ing thats applied to that Tests are just one form of of machines.
code before that ever gets feedback, and there are some But changes to the website, you get feedback lots of dif-
checked into the main ferent ways. You get feedback by testing it manually; you get
build is probably signii-
really good things about them, feedback by using it internally. You get feedback by rolling it to
cantly greater than at typi- but depending on the a small percentage of the servers and then watching the met-
cal business sites. Is that situation youre in, there rics, and if something goes haywire, then you just turn it of.
true, too? Binstock: So nonreversible decisions get the heavy rigor and
Beck: That is not true as a
can also be some very perhaps extreme testing, and everything else rides much
blanket statement. Theres substantial costs. more lightly in the saddle because of the reversibility.
a principle I learned from Beck: Yes.
an economics professor
called reversibility. Say Development of JUnit
you have a complicated system that reacts unpredictably to Binstock: Lets discuss the origins of JUnit. This has been doc-
stimuli. Henry Ford built these unprecedentedly large facto- umented a lot in various videos that youve made. So rather
ries, complicated systems in which a tiny little change could than go through it again, let me ask a few questions. How was
have huge efects. So his response to that was to reduce the the work initially divided between you and Erich Gamma?
number of states the factory could be in by, for example, Beck: We pair-programmed everything.
making all cars black. All cars werent black because Henry Binstock: So you guys were both involved throughout the
Ford was a controlling tightwad. It was simply so that the entire project?
paint shop either had paint or it didnt. Beck: We literally did not touch the code unless we were both
That made the whole thing easier to manage. Well, sitting togetherfor several years.
Facebook cant reduce the number of states Facebook is in. Binstock: Were you using a form of TDD at the time?
We want to keep adding more and more states. Thats how Beck: Yes, strictly. We never added a feature without a broken
we connect the world. So instead of reducing the number of test case.
states, we make decisions reversible. In Henry Fords factory, Binstock: OK. So how did you run the tests prior to JUnit being
once you cut a piece of metal, you cant uncut it. Well, we do able to run tests?
the equivalent of that all the time at Facebook. If you make Beck: By bootstrapping. It looked ugly at irst. You might be
a decision reversible, then you dont need to test it with the working from the command line, and then very quickly, you
kind of rigor that youre talking about. You need to pay atten- get enough functionality that it becomes convenient to run
tion when it rolls out and turn it of if it causes problems. the tests. Then every once in a while, you break things in a
Binstock: Thats an interesting alternative approach. way that gives you a false positive result, and then you say,
Beck: Well, theres a bunch of counterexamples. For example, All the tests are passing, but were not running any tests
code that handles money does go through extraordinary because of whatever change we just made. Then you have
rigor, or the Linux kernel goes through extraordinary rigor to go back to bootstrapping. People should try that exer-
37
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
cise. That is an extremely informative exer- if the latency matters and the half-life of
cise, to bootstrap a testing framework test the line of code is short, then you shouldnt
using itself. write tests.
Binstock: The only thing you had before that Binstock: Indeed, when exploring, if I run into
was SUnit [a JUnit precursor written by Beck errors, I may backtrack and write some tests
for Smalltalk]. That didnt seem like it was just to get the code going where I think its
going to be very helpful in writing JUnit supposed to go.
except on a conceptual level. Beck: I learned there are lots of forms of feed-
Beck: No, no, we started over from scratch. back. Tests are just one form of feedback,
Binstock: What role you did you have in and there are some really good things about
JUnit 5? As I understand it, you were not sig- them, but depending on the situation youre
niicantly involved in this release. in, there can also be some very substantial
Beck: Yes, I think no involvement whatso- costs. Then you have to decide, is this one of
ever is probably the closest. Actually, I think at one point, these cases where the trade-of tips one way or the other?
they were talking about how to make two diferent kinds of People want the rule, the one absolute rule, but thats just
changes in one release, and I said, by making them two difer- sloppy thinking as far as Im concerned.
ent releases. So one piece of parental advice, and that was it. Binstock: Yes, I think perhaps one of the great beneits of
more than two decades of programming experience is the
Test First great distrust in one overarching rule thats unlinchingly
Binstock: Do you still work on strictly a test-irst basis? and unbendingly applied.
Beck: No. Sometimes, yes. Beck: Yes. The only rule is think. IBM had that right.
Binstock: OK. Tell me how your thoughts have evolved on Binstock: I recall you saying that certain things, like getters
that. When I look at your book Extreme Programming Explained, and setters, really dont have to be written test irst.
there seems to be very little wiggle room in terms of that. Beck: I think thats more speciic than what I said. Its always
Has your view changed? been my policy that nothing has to be tested. Lots of people
Beck: Sure. So theres a variable that I didnt know existed at write lots of code without any tests, and they make a bunch
that time, which is really important for the trade-of about of money, and they serve the world. So clearly, nothing has to
when automated testing is valuable. It is the half-life of the be tested.
line of code. If youre in exploration mode and youre just There are lots of forms of feedback, and one of the fac-
trying to igure out what a program might do and most of tors that goes into the trade-of equation for tests is: whats
your experiments are going to be failures and be deleted in a the likelihood of a mistake? So if you have a getter, and its
matter of hours or perhaps days, then most of the beneits of just a getter, it never changes. If you can mess that up, we
TDD dont kick in, and it slows down the experimentationa have to have a diferent conversation. Tests are not going to
latency between I wonder and I see. You want that time ix that problem.
to be as short as possible. If tests help you make that time Binstock: When you originally formulated the rules for TDD,
shorter, ine, but often, they make the latency longer, and one of the cornerstones was that each iteration should have
38
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
the smallest possible incre- make thinner slices, and you can always think of more orders
ment of functionality. Where did Literate programs in which to implement things to serve diferent purposes.
that view come from? What was just dont maintain If you tell me I have a demo on Friday, I implement
important about the smallest things in a diferent order than if you tell me I have to run a
possible increment?
very well because load test on Friday, same project. Im going to slice it difer-
Beck: If you have a big, beautiful theres so much ently, and Im going to implement the slices in a very difer-
Italian salami, and you want to coupling between the ent order depending on what my next goal is.
know how long its going to take Binstock: So the smallest possible increment is a general
to eat the whole thing, an efective
prose and diagrams rule to apply when there are other factors that dont suggest
strategy is to cut of a slice and eat and the code. thicker slices?
it and then do the arithmetic. So, Beck: I dont believe in the smallest possible slice. I believe
1 millimeter takes me 10 seconds, in iguring out how to make the slices smaller. As small
then 300 millimeters are going to as I think Ive gotten the slices, I always ind some place,
take me 3,000 secondsnow maybe more, maybe less. There some way to make them half that size, and then I kick
may be positive feedback loops, negative feedback loops, myself: Why didnt I think of this before? Thats not one
other things to change that amount of time, but at least you test case; thats three test cases. Then progress goes much
have some experience with it. more smoothly.
The diference between a journeyman programmer and a Binstock: Well, if it hews very closely to the single respon-
master, from my perspective, is that the master never tries to sibility principle, it seems to me that you could have the
eat the whole salami at once. Number one, they always take same dynamic there, where methods do just very, very small
some big thing, and they put it into slices. Thats the irst operations, and then you have to string together thousands
skilliguring out where you can slice it. of little tiny BB-sized methods, and igure out how to put
The second skill is being creative about the order in them together.
which you consume the slices, because you might think you Beck: That would be bad because if you had to open 40 or 50
have to go left to right, but you dont. Somebody says, Well, classes, I would argue that violated cohesion at some point,
you have to write the input code before you can write the and thats not the right way to factor it out.
output code. I say, I respectfully disagree. I can build a data Binstock: I think were heading the same waythat there is a
structure in memory and write the output code from that. So limit at which cutting the slice even thinner doesnt add value
I can do input and then output, or output and then input. and starts to erode other things.
If I have n slices, I have n-factorable permutations of Beck: Well, today, and then I sleep on it, and I wake up in
those slices, some of which dont make any sense but many the morning and go, Why didnt I think of that? If I slice
of which do. So the two skills of the master programmer are it sideways instead of up and down, clearly it works bet-
slicing thinner slices and considering more permutations of ter. Something about walking out the door on Friday
the slices as the order of implementation. Neither of those afternoon and getting in my car to go homethat was the
skills ever reaches any kind of asymptote. You can always trigger for me.
39
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
The Coding Process theres a certain part of me
Binstock: Some years ago, I heard you recommend that when Im a big believer in that has a new appreciation
developers are coding, they should keep a pad of paper and getting rid of textual for things like Knuths Literate
a pen beside them and write down every decision that they Programming where you can
make about the code as theyre doing it.
source code and actually, in the comments, cap-
You suggested that we would all be startled by the num- operating directly on the ture what it is youre doing, what
ber of entries we would make, and that is exactly what hap- abstract syntax trees. youre trying to do, and what
pened with me. The more I looked at those lists that I put decisions youve made about it.
together, the more I realized that when interruptions occur, Actually, I worked that way for a
the ability to reconstruct the entire world view that I had while after hearing your discus-
before the interruption occurred depends a lot on being able sion of this particular discipline. In some ways, it was help-
to remember all of these microdecisions that have been ful. In other ways, it created a lot of clutter that ultimately I
made. The longer the gap, the more diicult it is even to con- had to go back and pull out of the comments. So the only rea-
sult the list to get those microdecisions back. son I brought it up was just to see if you had gone any further
Im wondering, have your thoughts on recording those with that.
microdecisions evolved in any way in which you can make Beck: What I ind with literate programs is they just dont
that list useful rather than just having it be an exercise in maintain very well because theres so much coupling between
coding awareness? the prose and diagrams and the code. Ive never put it in
Beck: No, no, it hasnt. One of the thingsand Ive written those terms before, but thats exactly right. If I make a
about thisis that Im having memory problems. So I have change to the code, not only do I have to change the code and
trouble holding big complicated, or even small, programs maybe the test, but I also have to change these four para-
in my head. I can be a pair-programming partner just ine graphs and those two diagrams, and I have to regather this
because I can rely on my partners memory, but me sitting data and render it into a graph again. So its not eiciently
down and trying to write a big complicated program is just maintainable. If you had code that was very stable and you
not something I can do anymore. wanted to explain it, then that wouldnt come into play, and it
I can still program, though, on the UNIX command line would make sense again.
because I can see the whole thing. So as long as its a one- Binstock: I had a conversation with Ward Cunningham in
liner and I can build it, like, one command at a time, then I which he talked about pair programming with you many
can accomplish programming tasks, but its not maintainable years ago and how surprised he was by how frequently you
code. Its all one-of codes. I do a lot of data mining. So if you guys would come to a decision point and the tool by which you
said, build a service to do X, thats just notdiferent people moved forward was by asking, What is the simplest possible
age in diferent ways at diferent rates, and so onbut thats thing we can do that will get us past this particular point? If
just not something that I can independently take of and do you always work from the simplest possible thing, do you not,
anymore, which is frustrating as hell. at some point, have to go back and refactor things so that you
Binstock: As I think about what youre talking about and I have code that you can be proud of rather than code that short-
think about my own eforts to record those microdecisions, circuits the problem? How do you balance those two things?
40
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
Beck: Sure. So I dont get paid to be proud. Like in JUnit, we Beck: You have to say, Well, yes, I dont know how tocon-
wrote code that wed be proud of or we didnt write the code. gratulations! You do know. I dont. So what am I supposed to
We could make that trade-of because we had no deadlines do? I cant pretend Im you.
and no paying customers.
But on a regular basis, if Im not proud of the code but my Testing Today
employer is happy with the results, yes. They call it work, and Binstock: Lets discuss microservices. It seems to me that
I get paid to do it. So there are other reasons to clean up. test-irst on microservices would become complicated in the
The answer is sure, youre going to make locally opti- sense that some services, in order to function, will need the
mized decisions because you just dont know stuf, and then presence of a whole bunch of other services. Do you agree?
you do know stuf and once you learn, then youre going to Beck: It seems like the same set of trade-ofs about having
realize the design shouldve been like this and this and this one big class or lots of little classes.
instead. Then you have to decide when, how, and whether to Binstock: Right, except I guess, here you have to use an awful
retroit that insight into your current code. Sometimes you do lot of mocks in order to be able to set up a system by which
and sometimes you dont. you can test a given service.
But I dont know what the alternative is. People say, Beck: I disagree. If it is in an imperative style, you do have
Well, arent you going to have to go refactor? Well, sure. So to use a lot of mocks. In a functional style where external
whats the alternative? dependencies are collected together high up in the call chain,
I remember I gave a workshop in Denmark, and I gave then I dont think thats necessary. I think you can get a lot of
a day-long impassioned speech about the beauties of itera- coverage out of unit tests.
tion. At the end of the day, this guy had been sitting in the Binstock: Today, the UI is so much more important than at
front row the entire day looking at me with an increasingly any previous time. How did you unit-test UIs in the past and
troubled expressionworse, and worse, and worse. He inally today? Were you using things like FitNesse and other frame-
raised his hand just before the time was up, and he said, works, or were you just eyeballing the results of the tests?
Wouldnt it be easier to do it right the irst time? I wanted Beck: I never had a satisfactory answer. Let me put it that
to hug him. I said, With all the compassion I have in me, yes, way. I tried a bunch of stuf. I built integration testing frame-
it would. I dont have any response other than that. works, I used other peoples tools, I tried diferent ways of
Binstock: Lovely question! summarizing what a UI looked like in some test-stable way,
Beck: I sat next to Niklaus Wirth on an airplane once. I talked and nothing worked.
to the agent. I told him we were colleagues and would he please Binstock: Today, youre pretty much in the same position,
move me, and so Im like a stalkerI fanboyed him. I dont arent you?
mind. If you get a chance to sit next to Niklaus Wirth, youre Beck: Yes, I havent seen anything that fundamentally
going to do it. So we got to talking, and I told him about TDD changes. Its all about false positives and false negatives.
and incremental design, and his response was, I suppose Whats the rate at which your tests say everythings OK, and
thats all very well if you dont know how to design software. everythings broken. That damages your trust in your tests.
Binstock: That sounds like the type of thing Wirth was known How often does the testing framework say somethings
to say. wrong, and everythings ine? Very often, one pixel changes
41
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
color very slightly, and Binstock: Well have to go through a number of iterations of
then the tests all break, If Im a little activated [when things like that before virtual reality actually inds a role
and you have to go through coding], then Ill listen to thatll help with the coding.
one at a time and go, Oh, Beck: Yes, Im a big believer in getting rid of textual source
yeah, no, this is ine. And
something soothing. And my code and operating directly on the abstract syntax trees. I
youre not going to do that go-to for that is Thomas did an experimental code editor called Prune with my friend
very many times before Tallis The Lamentations Thiago Hirai. It looked like a text editor and it rendered as
you just blow of the tests. a text editor would render, but you could only do operations
Binstock: The cost is the
of Jeremiah. on the abstract syntax trees, and it was much more eicient,
lost time and the lost trust. much less error-prone. It required far less cognitive efort.
Beck: Yes. That convinced me thats the wave of the future, and I dont
know if its going to be in 5 years or 25 years, but were all
Coding Environment going to be operating on syntax trees sometime soon.
Binstock: What does your preferred programming environ- Binstock: Yes, of all the things that have changed and moved
ment look like today, whether thats home or work? forward, the requirement that we still code at an ink-and-
Beck: The things I do that look like programming, I do paper level hasnt really moved forward very much.
entirely on either UNIX command line or in Excel. Beck: No, were coding on punch cards. Its rendered one on
Binstock: In Excel? top of the other, but its the same darn stuf.
Beck: Yes, because I can see everything. Binstock: The initial place of programmer activity hasnt
Binstock: How do you mean? evolved very much at all. Despite having wonderful IDEs and
Beck: So, like the transformations, I do the data transforma- things of that sort, the physical act is still very much the
tions, like numbers to numbers on the UNIX command line, same. One last thing, I know youre a musician. Do you listen
and then I render them into pictures using Excel. to music when you code?
Binstock: When youre programming things that are not Beck: Yes.
related to data mining, you mentioned earlier that you still Binstock: What kind of music do you ind that you enjoy most
use Smalltalk for exploratory things. coding to?
Beck: Yes, the advantage of Smalltalk for me is I memo- Beck: I use it to kind of regulate my energy level, so if Im
rized the API long enough ago that I still have access to all a little activated, then Ill listen to something soothing.
those details. And my go-to for that is Thomas Tallis The Lamentations of
Binstock: Do you typically work with multiple screens? Jeremiah, which is a very lowing vocal quartet kind of medi-
Beck: Yes, the more pixels, the better. It was a great Terry eval music. If Im a little low and I need picking up, then I
Pratchett quote, he says, People ask me why I have six listen to go-go music, which is an ofshoot of funk native to
screens attached to my Mac, and I tell them its because I Washington DC.
cant attach eight screens. Oculus or some kind of virtual Binstock: OK. Ive never heard of that.
reality is just going to blow that out of the water, but nobody Beck: Thats my upping music.
knows how. Binstock: Wonderful! Thank you! </article>
42
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
I f youve written any code in the last week, youve most likely
also written a unit test to go with it. Youre not alone. These
days, its rare to ind a codebase without unit tests. Many
the test suite? What if the safety net was full of holes? There
was also another related problem.
As the team members changed the code, they also
developers invest a lot of time in their tests, but are they needed to change the tests. Sometimes the team refactored
doing a good job? tests to make them cleaner. Sometimes tests had to be up-
This question began to trouble me seven years ago while dated in other ways as functionality was moved around the
I was working on a large legacy codebase for a inancial ser- codebase. So even if the tests were good at the point at which
vices company. The code was very diicult to work with, but they were written, how could the team be sure no defects
was a core part of the business and needed to be constantly were introduced into tests that were changed?
updated to meet new requirements. For the production code, the team had tests to catch mis-
A lot of my teams time was spent trying to wrestle the takes, but how were mistakes in the test code caught? Should
code into maintainable shape. This made the business owners the team write tests for the tests? If that were done, wouldnt
nervous. They understood that the team had a problem and the team eventually need to write tests for the tests for the
needed to make changes, but if the team introduced a bug it testsand then tests that tested the tests that tested the
could be very costly. The business owners wanted reassurance tests that tested the tests? It didnt sound like that would end
that everything was going to be all right. well, if it ever ended at all.
The codebase had a lot of tests. Unfortunately, the team Fortunately, there is an answer to these questions. Like
didnt need to examine them very closely to see that the tests many other teams, my team was using a code coverage tool to
were of no better quality than the code they tested. So, before measure the branch coverage of the tests.
the team members changed any code, they irst invested a lot The code coverage tool would tell which bits of the code-
of efort in improving the existing tests and creating new ones. base were well tested. If tests were changed, the team just
Because my team members always had good tests before had to make sure there was still as much code coverage as
they made a change, I told the business owners not to worry: before. Problem solved. Or was it?
if a bug were introduced while refactoring, the tests would There was one small problem with relying on code cover-
catch it. The owners money was safe. age in this way. It didnt actually tell the team anything about
But what if I were wrong? What if the team couldnt trust whether the code was tested, as I explain next.
43
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
Whats Wrong with Code Coverage? tistic go up. Fortunately, tests such
The problem is illustrated by some of the legacy tests I found as this are easy to ind using static You cant rely on
within the code. Take a contrived class such as this: analysis tools. code coverage tools
I also found tests like this:
class AClass {
to tell you that code
private int count; @Test has been tested.
public void testStuff() {
public void count(int i) { AClass c = new AClass();
if (i >= 10) { c.count(11);
count++; assert(c.currentCount() == 1);
} }
}
The programmer has used the assert keyword instead of a
public void reset() { JUnit assertion. Unless the test is run with the -ea lag set on
count = 0; the command line, the test can never fail. Again, bad tests
} such as this can be found with simple static analysis rules.
Unfortunately, these werent the tests that caused my
public int currentCount() { team problems. The more troubling cases looked like this:
return count;
} @Test
} public void shouldStartWithEmptyCount() {
assertEquals(0,testee.currentCount());
I might ind a test that looked like this: }
@Test @Test
public void testStuff() { public void shouldCountIntegersAboveTen() {
AClass c = new AClass(); testee.count(11);
c.count(11); assertEquals(1,testee.currentCount());
c.count(9); }
}
@Test
This test gives 100 percent line and branch coverage, but tests public void shouldNotCountIntegersBelowTen() {
nothing, because it contains no assertions. The test executes testee.count(9);
the code, but doesnt meaningfully test it. The programmer assertEquals(0,testee.currentCount());
who wrote this test either forgot to add assertions or wrote }
the test for the sole purpose of making a code coverage sta-
44
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
These tests exercise both branches of the code in the count Better Coverage with Mutation Testing
method and assert the values returned. At irst glance, One of the things I used to do after writing a test was double-
it looks like these are fairly solid tests. But the problem check my work by commenting out some of the code Id just
wasnt the tests the team had. It was the tests that the team implemented, or else Id introduce a small change such as
didnt have. changing <= to <, as in the earlier example. If I ran my test
There should be a test that checks what happens when and it didnt fail, that meant Id made a mistake.
exactly 10 is passed in: This gave me an idea. What if I had a tool that made
these changes automatically? That is, what if there were a tool
@Test that added bugs to my code and then ran the tests? I would
public void shouldNotCountIntegersOfExcelty10() { know that any line of code for which the tool created a bug
testee.count(10); but the test didnt fail was not properly tested. Id know for
assertEquals(0,testee.currentCount()); certain whether my test suite was doing a good job.
} Like most good ideas, it turned out that I wasnt the
irst to have it. The idea had a namemutation testingand
If this test doesnt exist, a bug could accidentally be intro- it was irst invented back in the 1970s. The topic had been
duced, such as the one below: researched extensively for 40 years, and the research com-
munity had developed a terminology around it.
public void count(int i) { The diferent types of changes I made by hand are called
if (i > 10) { // oops, missing the = mutation operators. Each operator is a small, speciic type of
count++; change, such as changing >= to >, changing a 1 to a 0, or com-
} menting out a method call.
} When a mutation operator is applied to some code, a
mutant is created. When tests are run against a mutant ver-
A small bug like this could have cost the business tens of sion of the code, if one of the tests fails, the mutant was
thousands of dollars every day it was in production until the killed. If no tests fail, the mutant survived.
moment that it was noticed and ixed. The academics examined the various types of muta-
This kind of problem cant be found by static analysis. It tion operators that were possible, looked at which were more
might be found by peer review, but then again it might not. In efective, and explored how well test suites that detected
theory, the code would never be written without a test if test- these artiicial bugs could detect real bugs. They also pro-
driven development (TDD) were used, but TDD doesnt magi- duced several automated mutation testing tools, including
cally stop people from making mistakes. some for Java.
So you cant rely on code coverage tools to tell you that So why havent you heard about mutation testing before?
code has been tested. Theyre still useful, but for a slightly Why arent all developers using mutation testing tools? Ill
diferent purpose. They tell you which bits of code are dei- talk about one problem now and the other a little later on.
nitely not tested. You can use them to quickly see which code The irst problem is straightforward: mutation testing is
deinitely has no safety net, in case you wish to change it. computationally very expensive. In fact, its so expensive that
45
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
until 2009, most academic research I wondered if I could do better. I already had some code
looked only at toy projects with less Its not possible that seemed like it might helpit was code for running
than a hundred lines of code. To to perform tests in parallel. It ran tests in diferent class loaders so that
understand why its so expensive, lets when state that was stored in static variables in legacy code
look at what a mutation testing tool
a mutation was changed, the running tests wouldnt interfere with each
needs to do. test when you other. I called it Parallel Isolated Test (or PIT).
Imagine you are performing have a failing After many evenings of experimentation, I managed to
mutation tests on the Joda-Time do better. My PIT mutation testing tool could analyze 10,000
library, which is a small library for
test, because mutations in the Joda-Time library in about three minutes.
handling dates and times. This library doing that would
has about 68,000 lines of code and mistakenly appear Introducing Pitest
about 70,000 lines of test code. It My tool has kept the initials of the codebase from which it
takes about 10 seconds to compile the
to kill any mutants grew, but its now also known as pitest, and it is used all
code and about 16 seconds to run the that it covered. over the world.
unit tests. Its used for academic research and for some exciting
Now, imagine that your mutation safety-critical projects, such as for testing the control sys-
test tool introduces a bug on every tems of the Large Hadron Collider at CERN. But mainly it is
seventh line of code. So youd have about 10,000 bugs. Each used to help test the kind of nonsafety-critical code that most
time you change a class to introduce a bug, you need to com- developers produce every day. So how does it manage to be so
pile the code. Perhaps that would take one second. So that much faster than the earlier system?
would be 10,000 seconds of compilation time to produce the First, it copied a trick from Jumble. Instead of spending
mutations (this is called the generation cost), which is more two and a half hours compiling source code, pitest modiies
than two and a half hours. You also need to run the test suite bytecode directly. This allows it to generate hundreds of
for each mutant. Thats 160,000 seconds, which is more than thousands of mutants in subsecond time.
44 hours. So performing mutation testing on the Joda-Time But, more importantly, it doesnt run all the tests against
library would take almost two days. each mutant. Instead, it runs only the tests that might kill a
Many of the early mutation testing tools worked exactly mutant. To know which tests those might be, it uses cover-
like this hypothetical tool. You still occasionally ind people age data.
trying to sell tools that work like this, but using such a tool is The irst thing pitest does is collect line-coverage data
clearly not practical. for each test so that it knows which tests execute which lines
When I became interested in mutation testing, I looked at of code. The only tests that could possibly kill a mutant are
the available open source tools. The best one I could ind was the ones that exercise the line of code the mutation is on.
Jumble. Its faster than the simplistic tool I described above, Running any other tests is a waste of time.
but it was still quite slow. And it had other problems that Pitest then uses heuristics to select which of the cover-
made it diicult to use. ing tests to run irst. If a mutant can be killed by a test,
46
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
pitest usually inds the killing test Lets go through it step by step.
in one or two attempts. The most efective <threads>2</threads> tells pitest to use two threads
The biggest speedup is time to perform when performing mutation testing. Mutation testing usually
achieved when you have a mutant scales well, so if you have more than two cores, it is worth
that is not exercised by any test.
mutation tests on increasing the number of threads.
With the traditional approach, your code is when you <timestampedReports>false</timestampedReports> tells
youd have to run the entire test write the code. pitest to generate its reports in a ixed location.
suite to determine that the mutant <mutators><value>STRONGER</value></mutators> tells
could not be killed. With the pitest to use a larger set of mutation operators than the
coverage-based approach, you can default. This section is commented out in the POM ile at the
determine this instantly with almost no computational cost. moment. Ill enable it a little later on. If youre just starting
Line coverage identiies code that is deinitely not tested. out with mutation testing on your own project, I suggest you
If there is no test coverage for the line of code where a mutant also stick with the defaults at irst.
is created, then none of the tests in the suite can possibly kill The pitest Maven plugin assumes that your project follows
it. Pitest can mark the mutant as surviving without doing any the common convention of having a group ID that matches
further work. your package structure; that is, if your code lives in packages
named com.mycompany.myproject, it expects the group ID to
Using Pitest be com.mycompany.myproject. If this is not the case, you might
Setting up pitest for your project is straightforward. IDE plug- get an error message such as the following when you run pitest:
ins have been built for Eclipse and IntelliJ IDEA, but person-
ally I prefer to add mutations from the command line using No mutations found. This probably means there is an issue
the build script. Some very useful features of pitest are acces- with either the supplied classpath or filters.
sible only in this way, as youll see in a moment.
I normally use Maven as my build tool, but pitest plugins Google Truths group name doesnt match the package struc-
also exist for Gradle and Ant. ture, so I added this section:
Setting up pitest for Maven is straightforward. I usually
bind pitest to the test phase using a proile named pitest. <targetClasses>
Then pitest can be run by activating the proile with P, as <param>com.google.common.truth*</param>
shown here: </targetClasses>
mvn -Ppitest test Note the * at the end of the package name.
Pitest works at the bytecode level and is conigured by
As an example, Ive created a fork of the Google assertion supplying globs that are matched against the names of the
library Truth on GitHub, and I added pitest to the build. You loaded classes, not by specifying the paths to source iles.
can see the relevant section of the project object model (POM) This is a common point of confusion for people using it for
ile here. the irst time.
47
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
Another common problem when setting up pitest for the make them clean up after themselves when they inish. But
irst time is this message: All tests did not pass without by far the best approach is to avoid having shared mutable
mutation when calculating line coverage. Mutation test- state in your program in the irst place.
ing requires a green suite. Finally, the setup for using the Google Truth library
This message can occur if you have a failing test. Its not includes this section:
possible to perform a mutation test when you have a failing
test, because doing that would mistakenly appear to kill <excludedClasses>
any mutants that it covered. Sometimes youll also get the <param>
message when all the tests pass when run normally with *AutoValue_Expect_ExpectationFailure
mvn test. If this happens, there are a few possible causes. </param>
Pitest tries to parse the coniguration of the Sureire test </excludedClasses>
runner plugin and convert the coniguration to options that
pitest understands. (Sureire is the plugin that Maven uses by This coniguration prevents all classes whose name ends in
default to run unit tests. Often no coniguration is required, AutoValue_Expect_ExpectationFailure from having muta-
but sometimes tests need some special coniguration in order tions seeded into them. These classes are autogenerated by
to work, which must be supplied in the pom.xml ile.) the Google Truth build script. There is no value in performing
Unfortunately, pitest cant yet convert all the possible mutation testing on them, and any mutations that are created
types of Sureire coniguration. If your tests rely on system would be diicult to understand because you do not have the
properties or command-line arguments being set, you need source code.
to specify them again in the pitest coniguration. Pitest also provides other ways to exclude code from being
Another problem thats more diicult to spot is order mutation-tested. Details can be found on the pitest website.
dependencies in the tests. Pitest runs your tests many times
in many diferent sequences, but you might have a test that Understanding the Pitest Report
fails if certain other tests run before it. Lets do a sample run and look at the result it generates. To
For example, if you have a test called FooTest that sets begin, check out the source code for the Google Truth library,
a static variable in a class to false, and you have another test and run pitest using Maven:
called BarTest that assumes that the variable is set to true,
BarTest will pass if it is run before FooTest but fail if it is run mvn -Ppitest test
afterward. By default, Sureire runs tests in a random but
ixed order. The order changes when a new test is added, but It should take about 60 seconds once Maven has inished
you might never have run the tests in an order that reveals downloading the dependencies. After the run, youll ind
the dependency. When pitest runs the tests, the order it uses an HTML report in the target/pitReports directory. For
might reveal the order dependency for the irst time. the Truth project, youll ind the report under core/target/
Test-order dependencies are very hard to spot. To avoid pitReports.
them, you can make tests defensively set shared state on The pitest report looks very similar to the reports that
which they depend to the right value when they start, and standard coverage tools produce, but it contains some extra
48
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//junit 5 /
information. Each package is listed with its overall line cov- the report: a list of all the tests that were used to challenge
erage and its mutation score shown side by side. the mutants in this ile and how long each of them took to
You can drill down into each source ile to get a report run. Above this is a list of all the mutations. If you hover over
such as the one shown in Figure 1. them, youll see the name of the test that killed the mutant.
Line coverage is shown as blocks of color that span the Google Truth was developed without using pitest or any
width of the page. Green indicates that a line is executed by other mutation testing tool and, on the whole, the team that
tests, and red indicates that it is not. developed it did a very good job. A mutation testing score of
The number of mutations created on each line is shown 88 percent is not easy to achieve. But still, there are holes.
between the line number and the code. If you hover over the The most interesting mutants are the ones that appear
number, youll get a description of the mutations that were on the green lines that indicate they were covered by tests. If
created and their status. If all the mutants were killed, the a mutant was not covered by a test, it is not surprising that it
code is shown in darker green. If one or more of them sur- survived and does not give any additional information com-
vived, the code will be highlighted in red. pared to line coverage. But if a mutant was covered, you have
Theres additional useful information at the bottom of something to investigate.
54
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//new to java /
iterators because its less error-prone and more concise. The Template method
Java programmers toward embracing type inference). In this We show how lambda expressions can provide an alternative
article, we examine how lambdas can reduce the code needed way to solve the same problem for which each of these design
to implement several programming patterns. To follow along, patterns is intended.
youll need a basic familiarity with lambdas.
A speciic class of patterns is called design patterns. They Strategy Pattern
are a reusable blueprint, if you will, for a common problem The strategy pattern is a common solution for representing a
when designing software. Its a bit like how construction family of algorithms and letting you choose among them at
engineers have a set of reusable solutions to construct bridges runtime. You can apply this pattern to a multitude of scenar-
for speciic scenarios (such as suspension bridge, arch bridge, ios, such as validating an input with diferent criteria, using
and so on). For example, the visitor design pattern is a com- diferent ways of parsing, or formatting an input.
mon solution for separating an algorithm from a structure The strategy pattern consists of three parts, as illustrated
on which it needs to operate. Another pattern, the singleton in Figure 1.
pattern, is a common solution to restrict the instantiation of a These parts are:
class to only one object. An interface to represent some algorithm (the interface
to the problems the design patterns are tackling but often to represent multiple algorithms (the concrete classes
with less work and in a simpler way. Many existing object- ConcreteStrategyA and ConcreteStrategyB)
oriented design patterns can be made redundant or written in One or more clients that use the strategy objects
55
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//new to java /
public class Validator{
ConcreteStrategyB
Strategy private final ValidationStrategy strategy;
Client
+ execute ()
ConcreteStrategyA public Validator(ValidationStrategy v){
this.strategy = v;
Figure 1. The strategy design pattern
}
Lets say youd like to validate whether text input is public boolean validate(String s){
properly formatted for diferent criteria (for example, it return strategy.execute(s);
consists of only lowercase letters or is numeric). You start }
by deining an interface to validate the text (represented as }
a String):
Then with this code, the irst example returns false, the
public interface ValidationStrategy { second one true:
boolean execute(String s);
} Validator v1 =
new Validator(new IsNumeric());
Second, you deine one or more implementation(s) of System.out.println(v1.validate("aaaa"));
that interface:
Validator v2 =
public class IsAllLowerCase new Validator(new IsAllLowerCase());
implements ValidationStrategy { System.out.println(v2.validate("bbbb"));
public boolean execute(String s){
return s.matches("[a-z]+"); You should recognize that ValidationStrategy is a functional
} interface. This means that instead of declaring new classes to
} implement diferent strategies, you can pass lambda expres-
sions directly, which are more concise:
public class IsNumeric
implements ValidationStrategy { // with lambdas
public boolean execute(String s){ Validator v3 =
return s.matches("\\d+"); new Validator((String s) ->
} s.matches("\\d+"));
} System.out.println(v3.validate("aaaa"));
As you can see, lambda expressions remove the boilerplate The processCustomer method provides a sketch for the
code inherent in the strategy design pattern. If you think online banking algorithm: fetch the customer given his or
about it, lambda expressions encapsulate a piece of code (or her ID and then make the customer happy. Diferent branches
strategy), which is what the strategy design pattern was cre- can now provide diferent implementations of the method
ated for, so we recommend that you use lambda expressions makeCustomerHappy by subclassing the OnlineBanking class.
instead for similar problems. You can tackle the same problem (creating an outline of
an algorithm and letting implementers plug in some parts)
Template Method Pattern using your favorite lambdas. The diferent components of the
The template method design pattern is a common solution algorithms you want to plug in can be represented by lambda
when you need to represent the outline of an algorithm and expressions or method references.
have the additional lexibility to change certain parts of it. Here, we introduce a second argument to the method
In other words, the template method pattern is useful when processCustomer of type Consumer<Customer> because it
you ind yourself in a situation such as Id love to use this matches the signature of the method makeCustomerHappy
algorithm but I need to change a few lines so it does what deined earlier:
I want.
Lets look at an example of how this pattern works. public void processCustomer(
Say you need to write a simple online banking application. int id, Consumer<Customer> makeCustomerHappy){
Users typically enter a customer ID, and then the applica- Customer c = Database.getCustomerWithId(id);
tion fetches the customers details from the bank database makeCustomerHappy.accept(c);
and inally does something to make the customer happy. }
Diferent online banking applications for diferent bank-
ing branches may have diferent ways of making a customer You can now plug in diferent behaviors directly without
happy (for example, adding a bonus on the account or just subclassing the OnlineBanking class by passing lambda
sending less paperwork). You can write the following abstract expressions:
class to represent the online banking application:
new OnlineBankingLambda().processCustomer(1337,
abstract class OnlineBanking { (Customer c) -> System.out.println(
"Hello " + c.getName());
public void processCustomer(int id){
Customer c = Database.getCustomerWithId(id); This is another example of how lambda expressions can help
makeCustomerHappy(c); you remove the boilerplate inherent in design patterns!
}
57
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//new to java /
Observer Pattern You can now declare diferent observers (here, the three
The observer design pattern is a common solution when newspapers) that produce a diferent action for each diferent
an object (called the subject) needs to automatically notify keyword contained in a tweet:
a list of other objects (called observers) that some event has
occurred (for example, a state change). You typically come class NYTimes implements Observer{
across this pattern when working with GUI applications. You public void notify(String tweet) {
register a set of observers on a GUI component such as a if(tweet != null && tweet.contains("money")){
button. If the button is clicked, the observers are notiied System.out.println(
and can execute a speciic action. But the observer pat- "Breaking news in NY! " + tweet);
tern isnt limited to GUIs. For example, the observer design }
pattern is also suitable in a situation where several traders }
(observers) might wish to react to the change of price of a }
stock (subject). Figure 2 illustrates the UML diagram of the class Guardian implements Observer{
observer pattern. public void notify(String tweet) {
Lets write some code to see how the observer pattern if(tweet != null && tweet.contains("queen")){
is useful in practice. Youll design and implement a custom- System.out.println(
ized notiication system for an application like Twitter. The "Yet more news in London... " + tweet);
concept is simple: several newspaper agencies (The New York }
Times, The Guardian, and Le Monde) are subscribed to a feed of }
news tweets and may want to receive a notiication if a tweet }
contains a particular keyword. class LeMonde implements Observer{
First, you need an Observer interface that groups the public void notify(String tweet) {
diferent observers. It has just one method called notify if(tweet != null && tweet.contains("wine")){
that will be called by the subject (Feed) when a new tweet is System.out.println(
available: "Today cheese, wine, and news! " + tweet);
}
interface Observer { }
void notify(String tweet); }
}
Youre still missing the crucial part: the subject! Lets deine
an interface for it:
ConcreteObserverB
Subject Observer
interface Subject{
+ notifyObserver() + notify() void registerObserver(Observer o);
ConcreteObserverA void notifyObservers(String tweet);
Figure 2. The observer design pattern }
58
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//new to java /
The subject can register a new observer using the register all providing implementation for a single method: notify.
Observer method and notify his observers of a tweet with the Theyre all just wrapping a piece of behavior to execute when
notifyObservers method. Lets go ahead and implement the a tweet arrives. Lambda expressions are designed speciically
Feed class: to remove that boilerplate. Instead of instantiating three
observer objects explicitly, you can pass a lambda expression
class Feed implements Subject{ directly to represent the behavior to execute:
Contribute to Java by
Adopting a JSR
How you can build your developer skills by contributing to Java standards
MARTIJN VERBURG
Speciication design
Adopt-a-JSR is focused on getting Java developers involved Technology compatibility kit work
with emerging Java standards as early as possible. Individual If youd like to help Java continue to be the leading language
developers, Java user groups, and organizations are all eligible and platform choice for software engineers, then this is a
and welcome to join. great place to start.
The Java standards covered by JSRs determine what goes Many potential contributors are reluctant to participate
into Java SE, Java EE, and Java ME. For example, Project Jigsaw, in a JSR because theyre not experts. But JSRs should be seen
which provides a modularization platform for Java SE, is more as special interest groups (SIGs) on a particular topic, in
deined in JSR 376. For Java EE, the Servlet 4.0 speciication which the members fulill a wide variety of diferent needs.
is determined in JSR 369. Many of the tasks listed above require you only to have an
interest in the topic and the willingness to spend some time.
Why Participate in Adopt-a-JSR? Theyre not reserved for domain experts.
The main reason for this program is to ensure that the emerg-
ing standards for the Java ecosystem are technically sound, Get Started
relevant, usable, accessible, and free from intellectual property To get started, join the community at Adopt-a-JSR. There are
entanglements. With an estimated 10 million developers using links on that portal for signing up for the IRC channel, mail-
PHOTOGRAPH BY BOB ADLER Java, this kind of attention to emerging standards is important. ing lists, and so forth. The next step is to pick an active JSR
61
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//community participation /
that you and your colleagues are Evangelize the JSR through social media, blogging, and
personally interested in and pas- If youve wondered how talks
sionate about. You can then post Java SE, Java EE, and Arrange hack days and meetups to work with, or test out,
advice on how to join the oicial and wish you were contribute to discussions on the mailing list
JSR youre interested in and how able to help, then Help build the reference implementation or technical com-
66
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//ix this /
Quiz Yourself
Would you ever use a finally clause with a try-with-resources? Explore this and
other subtle questions from an author of the Java certification tests.
SIMON ROBERTS
O nce again, Ive composed more problems that simulate
questions from the 1Z0-809 Programmer II exam, which
is the certiication test for developers who have been certi-
d. An exception at line n1
e. An exception at line n2
ied at a basic level of Java 8 programming knowledge and Question 2. Given this code:
now are looking to demonstrate more-advanced expertise. Collection<String> coll = new ArrayList<>();
[Readers looking for basic instruction should consult the New coll.add("Fred"); coll.add("Jim"); coll.add("Sheila");
to Java column, which appears in every issue. Ed.] As usual, System.out.println("coll is " + coll);
these questions can require careful deduction to obtain the coll.remove(0); // line n1
right answer. System.out.println("coll is " + coll);
Question 1. Given the following code: What is the result? Choose one.
ResourceBundle properties = ResourceBundle.getBundle( a. coll is [Fred, Jim, Sheila]
"scratch.ConfigData", Locale.FRANCE); // line n1 coll is [Jim, Sheila]
b. coll is [Fred, Jim, Sheila]
System.out.println( coll is [Fred, Jim, Sheila]
properties.getString("help")); // line n2 c. Compilation fails at line n1.
d. An exception is thrown at line n1.
And a directory and text ile scratch/ConfigData.properties
located on the CLASSPATH of the running program, which Question 3. Given that the current working directory of the
contains the following text: program is empty, and given this code fragment:
file=File
edit=Edit Path p = Paths.get("a", "b", "cee"); // line n1
help=Help System.out.println(p.endsWith(Paths.get("b", "cee")));
System.out.println(p.endsWith(Paths.get("ee")));
What is the result? Choose one.
a. help What is the result? Choose one.
b. Help a. true
c. Aide true
67
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016
//ix this /
b. true b. three
false two
c. false one
true c. three
d. false one
false two
e. An exception at line n1 d. two
one
Question 4. Given the following code: three
class MyResource implements AutoCloseable { e. The words one, two, and three are all output, but the
private String name; order is not determined.
public MyResource(String name) {
this.name = name; Question 5. Given this code:
} class MyJobException extends Exception {}
magazine
By and for the Java community
74
ORACLE.COM/JAVAMAGAZINE /////////////////////////////// NOVEMBER/DECEMBER 2016