International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013

TDD for Embedded Systems: Feasibility and Issues

Ashish Soni#1, Nikhil Saxena#2, Narayan Sharma#3,Sapan Kumar Gupta#4
#

Electronics and Communication Deptt. JIIT, Noida following list are based on Kent Becks description in his book Test-Driven Development. Add a small test. Run all the tests and see the new one fail, maybe not even compile. Make the small changes needed to pass the test. Run all the tests and see the new one pass. Refactor to remove duplication and improve expressiveness.

Abstract The paper provides a deep insight regarding the test driven development (TDD). The paper discusses the challenges and some solutions to adapt it in embedded system. Increasing complexity and the need to forever-shorter development cycles are pushing embedded development teams to the limit. To gain a competitive edge, companies are looking towards agile methods, such as Test Driven Development to help prevent defects by ensuring requirements are understood at the earliest stage. TDD has not yet applied to the embedded applications widely and therefore embedded system designers face a lot of problems. The paper provides a good awareness regarding the use of TDD in embedded system. The paper also deals with these issues and their solution through test driven development. Keywords TDD, microcycle, MCH, Dual Target Testing, Testing with hardware

I. INTRODUCTION Test-Driven Development is a technique for building software incrementally. Simply put, no production code is written without first writing a failing unit test. Tests are small. Tests are automated. Test-driving is logical. Instead of diving into the production code, leaving testing for later, the TDD practitioner expresses the desired behavior of the code in a test. The test fails. Only then do they write the code, making the test pass. TDD is a part of a software development methodology called Extreme Programming. It stresses customer satisfaction. Instead of delivering everything engineers could possibly want on some date far in the future, this process delivers the software they need as they need it. Extreme programming empowers developers to confidently respond to changing customer requirements, even late in the life cycle. TDD began to receive publicity in the early twenty-first century as an aspect of Extreme Programming. TDD is based on the idea to create tests for the program before you develop the program code. This is the opposite of what is usual in current software development methodologies. The availability of tests before actual development ensures rapid feedback after any change. Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing. The TDD practice starts with thoughts on how to test the required functionality. After writing automated test cases that generally will not even compile, the programmers write implementation code to pass these test cases.

Figure 1. TDD Microcycle state machine flow

III. EMBEDDED TDD TECHNIQUE Although TDD has a lot of advantage but still embedded developers have some issues for its implementation

A. Dual Target Testing Dual targeting means to design a code considering both final target hardware and development system from starting. Dual-targeting solves several problems. It allows you to test code before the hardware is ready, and you can avoid the hardware bottleneck throughout the development cycle. You also avoid the finger pointing that goes with simultaneous hardware software debugging. It is a practice that keeps you moving fast. Due to differences between the development and target environments some risks are also inherited with it such II. TDD MICROCYCLE A TDD state machine simple process is shown in the figure. as runtime libraries may be different, include filenames and At the core of TDD is a repeating cycle of small steps known features may be different, primitive data types might have as the TDD microcycle. The steps of the TDD cycle in the

ISSN: 2231-5381

http://www.ijettjournal.org

Page 2141

International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
different sizes, Byte ordering and data structure alignments may be different. B. Testing With Hardware The use of interfaces and isolation allows us to develop and test significant system behavior without access to the real target. Automated tests to exercise the hardware are valuable to both hardware and software developers because they perform independent tests of hardware subsystems. Testing with hardware falls into the following categories: Automated hardware tests Partially automated hardware tests Automated hardware tests with external instrumentation There should be many tests like this focused on specific scenarios. Much of the core functionality of a system can be tested well before the target hardware is available. Furthermore, when the target becomes available, bring up time and subsequent integration and testing can be substantially reduced because the core system functionality has already been well tested. The isolation approach just described is not new; it simply comprises the application of the well-known practices of abstraction and encapsulation and the separation of concerns that are essential for TDD. This technique is independent of any computer language or tool; it is simply a matter of applying good modular design practices. In C++, polymorphism is exploited for creating interfaces between modules. In C or C++, we can use the linker to substitute in test stubs. An example for a home guard system is shown in the Appendix A. C. Embedded Model Conductor Hardware Example: MCH in a C based environment Creating mocks and tests in an embedded C environment is accomplished through compiled mock.o implementations of header file function declarations. For example, suppose hardware.h declares all functions for interfacing the hardware features of a particular microcontroller. In this example, Conductor tests will verify that the Conductor makes specific calls on the hardware with appropriate parameters. As such, a mock hardware.c definition file will be written containing otherwise empty functions that store individual function call parameter values or return specific values both as defined by global variables. Object files for mock and functional code are linked together, and tests access the previously mentioned global values to verify the Conductor calls to the mock hardware.h interface. The code blocks in Appendix B are examples drawn from a real world project developed for Savant Automation of Grand Rapids, Michigan. Savant builds Automated Guided Vehicles. These samples pertain to a dedicated speed control board; the functions shown set output drive voltage. In this testing scenario, we illustrate a Conductor under test. The example tests verify that the Conductor is correctly using the hardware interface. IV. CHALLENGES ISSUES AND SOLUTIONS Although TDD has a lot of advantage but still embedded developers have some issues for its implementation. To write all these test codes more time is needed. Its hard to shift from DLP to TDD, and this is a common reaction: Lets just write the tests after. This practice has a name, Test-After Development. But still TDD may be preferred. But in it, it is needed to maintain the tests. We get value from having the tests that makes the maintenance effort worth the investment. TDD will not prevent all bugs, but that does not make a case for not doing TDD. We will still need integration tests, acceptance tests, exploratory tests, and load tests. TDD will eliminate many of the problems so that the higher-level tests are finding appropriate problems. Integration tests should find integration problems, acceptance tests should show that the code meets its requirements, and load tests should help determine whether the system can meets its design limits. When changes occur, there will be implications at the unit test level, and the TDD tests will help assure that changes have only the intended consequences. Constrained memory is the reality for many embedded developers. Running tests in the development system wont reveal the same memory constraints found in the target. Use dual-targeting so the bulk of your code is tested off-target, Make a lab version of your target system with plenty of memory to hold all production code and test cases and finding small test harness may limit this problem upto an extent. When TDD is applied on a real project during the development of an embedded system application, then it is to be developed with custom hardware which takes a long time to be developed, nearly months. There are issues with timing, concurrent processing, and throughput. Also, there are many unknowns, as one has to settle a lot many of the architecture issues, including same code. This is very powerful as it helps to greatly reduce one of the software developer's biggest problems, side effect defects. The unit test suite for a module also provides an unambiguous detailed specification of the module under test. In contrast, acceptance tests operate on integrated groups of modules to show that the software meets its requirements .Modules are bound together and various test scenarios are fed into them, demonstrating the system or subsystem behavior. Ideally, nonprogrammers write these tests in an application-specific test language. These tests become part of the engineering specifications. This paper is mainly concerned with unit testing. But many developers and some people from managerial people think that these are just the crippling uncertainties, and hence think that the only productive work to be done was to write design documents in preparation for when one could start the implementation. To ones surprise, developers are actually able to start design and implementation within days on the areas in which the requirements are understood. One can employ TDD to test core behavior and use the best practices of object-oriented design to isolate areas of uncertainty. Work on requirements continued in parallel, reducing the uncertainty in the process. After very few months of development, one usually has only

ISSN: 2231-5381

http://www.ijettjournal.org

Page 2142

International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
two or three bugs that required more than a few minutes of debugging. The total time spent in debugging is approximately only about a tenth of the project duration. V. CONCLUSIONS The paper discusses about the issues and some solutions of implementation of TDD in the embedded application. TDD is important software development practices that can help embedded developers deliver higher quality products. The embedded TDD cycle can help take hardware availability off the software critical path, enabling steady progress with or without hardware. TDD can be used for embedded development in C and C++. Java may also be an option for some embedded systems. Applying TDD improves test coverage and makes it possible to find defects earlier in the development life cycle. Highly coupled designs and implementations, as found in many embedded systems, make automated test a huge challenge. To be testable, modules have to be independent because automated tests operate on independent modules. A commitment to TDD will result in lower coupling and higher cohesion, which are key attributes of solid designs. APPENDIX A The example code for the home guard system is shown: TEST(HomeGuard, WindowIntrusion) { MockAlarmPanel* panel = new MockAlarmPanel(); HomeGuard hg(panel); hg.arm(); hg.windowIntrusion(); CHECK(true == panel->isArmed()); CHECK(true == panel->isAudibleAlarmOn()); CHECK(true == panel->isVisualAlarmOn()); CHECK(panel->getDisplayString() == "Window Intrusion"); } APPENDIX B hardware.h /// Get feedback from analog drive output. millivolts Hardware_GetFeedbackVoltage(void); /// Set the drive output voltage void Hardware_SetOutputVoltage( millivolts output); /// Set the error flag. void Hardware_SetError(bool err); model.h typedef struct _ModelInstance { millivolts FeedbackVoltage; millivolts OutputVoltage; bool Error; } ModelInstance; /// Set the feedback voltage. void Model_SetFeedbackVoltage( millivolts feedback); /// Get drive output voltage for hardware. millivolts Model_GetOutputVoltage(void); /// Get the error state. bool Model_GetError(void); conductor.h /// Callback for hardware feedback voltage. void Conductor_HandleFeedbackVoltage(void); /// Control loop called by main() forever void Conductor_Run(void); mockhardware.h /// Feedback voltage to return extern millivolts Hardware_InputFeedbackVoltage; /// Output voltage set by conductor. extern millivolts Hardware_OutputDriveOutputVoltage; /// Error flag extern bool Hardware_OutputError; mockhardware.c #include "mockhardware.h" #include "hardware.h" #include "conductor.h" millivolts Hardware_InputFeedbackVoltage; millivolts Hardware_OutputDriveOutputVoltage; bool Hardware_OutputError; millivolts Hardware_GetFeedbackVoltage(void) { return Hardware_InputFeedbackVoltage; } void Hardware_SetOutputVoltage( millivolts output){ Hardware_OutputDriveOutputVoltage = output; } void Hardware_SetError(bool err) { Hardware_OutputError = err; } mockmodel.h /// Modeled system feedback voltage extern millivolts Model_FeedbackVoltage; /// Modeled system output voltage extern millovolts Model_OutputVoltage; /// Modeled error state extern bool Model_Error; mockmodel.c // Linked with testconductor.o in place of // model.o to allow conductor tests // independent of logic in actual model. #include "mockmodel.h" millivolts Model_FeedbackVoltage; millovolts Model_OutputVoltage; bool Model_Error; void Model_SetFeedbackVoltage( millivolts feedback) { Model_FeedbackVoltage = feedback; } millivolts Model_GetOutputVoltage(void) { return Model_OutputVoltage; }

ISSN: 2231-5381

http://www.ijettjournal.org

Page 2143

International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
bool Model_GetError(void) { return Model_Error; } model.c #include "model.h" ModelInstance Model; void Model_SetFeedbackVoltage( millivolts feedback) { Model.FeedbackVoltage = feedback; if(feedback != Model.OutputVoltage) { // realistically use nominal value Model.Error = true; } } millivolts Model_GetOutputVoltage(void) { return Model.OutputVoltage; } bool Model_GetError(void) { return Model.Error; } conductor.c #include "model.h" #include "hardware.h" void Conductor_HandleFeedbackVoltage(void) { Model_SetFeedbackVoltage( Hardware_GetFeedbackVoltage()); } void Conductor_Run(void) { Hardware_SetError(Model_GetError()); if(!Model_GetError()) { Hardware_SetOutputVoltage( Model_GetOutputVoltage()); } } testconductor.c #include "conductor.h" #include "mockhardware.h" #include "mockmodel.h" static void testHandleFeedback(void) { Hardware_InputFeedbackVoltage = 7; Conductor_HandleFeedbackVoltage(); TEST_ASSERT_EQUAL_INT( 7, Model_FeedbackVoltage); } static void testConductorRun(void) { Model_Error = false; Model_OutputVoltage = 78; Conductor_Run(); TEST_ASSERT_MESSAGE( Hardware_OutputError == false, "Error set incorrectly"); TEST_ASSERT_EQUAL_INT( 78, Hardware_OutputDriveOutputVoltage); Model_Error = true; Model_OutputVoltage = 99; Conductor_Run(); TEST_ASSERT_MESSAGE( Hardware_OutputError == true, "Error not set"); TEST_ASSERT_EQUAL_INT(78, Hardware_OutputDriveOutputVoltage); } ACKNOWLEDGMENT The authors thank the assistance provided by Sanjeev Kumar of NIT Kurukshetra, Rahul Singh M.P.E.C. Kanpur and Aditya Pathak of YLM Tube Solutions & Service India Pvt. Ltd. Additional thanks go to Himanshi Sharma of Sharda University, Ragini Rai of HCL Technologies, Rahul Pandey of M.P.E.C.Kanpur and Pushkar Mishra of TCS India Ltd. for their great efforts and inspiring me for this paper. REFERENCES
[1] [2] [3] S. M. Metev and V. P. Veiko, Laser Assisted Microtechnology, 2nd ed., R. M. Osgood, Jr., Ed. Berlin, Germany: Springer-Verlag, 1998. K. Beck, "Extreme programming eXplained: embrace change", Addison-Wesley, Reading (2000) L. Huang, M. Holcombe, "Empirical investigation towards the effectiveness of Test First programming", Inf. Softw. Technol. 51, 2009 O. Salo, P. Abrahamsson, "Empirical Evaluation of Agile Software Development: A Controlled Case Study Approach", Product Focused Software Process Improvement, 2004 A. Hfer, M. Philipp, "An Empirical Study on the TDD Conformance of Novice and Expert Pair Programmers", XP, 2009 M. M. Mller, O. Hagner, "Experiment about test-first programming", IEE Procedings-Software. v149, 002 E. M. Maximilien, L. Williams, "Assessing test-driven development at IBM", 25th international Conference on Software Engineering, ICSE 2003 B. George, L. Williams, "A Structured Experiment of Test-Driven Development", Information and Software Technology 46(5), pp. 337342, 2003 J. C. Sanchez, L. Williams, E. M. Maximilien, "On the Sustained Use of a Test-Driven Development Practice at IBM", AGILE, 2007 G. Canfora, A. Cimitile, F. Garcia, M. Piattini, C. A. Visaggio, "Evaluating advantages of test driven development: A controlled experiment with professionals", International Symposium on Empirical Software Engineering, 2006 L. Cao, B. Ramesh, "Agile Requirements Engineering Practices: An Empirical Study", Software, IEEE , vol.25, 2008 Curriculum",International Symposium on It in Medicine and Education, 2008 L. R. Chien, D. J. Buehrer, C. Y. Yang, C. M. Chen, "An Evaluation of TDD Training Methods in a Programming with Professionals", Product-Focused Software Process Improvement (PROFES), 2006 L.-O. Damm, L. Lundberg, "Results from introducing component level test automation and Test-Driven Development", Journal of Systems and Software, 2006 L.-O. Damm, L. Lundberg, "Quality impact of introducing fault detection effectiveness of unit test suites", Softw. Process 13, 2008 M. A. Domino, R. W. Collins, A. R. Hevner, "Controlled experimentation on adaptations of pair programming", Inf. Technol. and Management 8, 2007 D. Janzen, H. Saiedian, "Does Test-Driven Development Really Improve Software Design Quality?", IEEE Software, 25, 2008

[4]

[5] [6] [7]

[8]

[9] [10]

[11]

[12]

[13]

[14] [15]

[16]

ISSN: 2231-5381

http://www.ijettjournal.org

Page 2144