Ongoing innovation in the x86 semiconductor industry is the foundation for the near-ubiquitous use of x86 embedded computing technology in the ever-growing range of SFF applications. Even with continued improvements in CPU performance and power efficiency, however, designers of SFF portable systems remain challenged to achieve their most ambitious design goals for graphics performance and visual immersion. Growing demand for higher performance graphics capabilities has led OEMs to explore new x86 processor architectures that promise to meet exacting multimedia performance requirements for applications spanning commercial, medical, and industrial domains, with a growing focus on portable and/or battery-powered devices.

Embedded boards and modules equipped with new-generation Accelerated Processing Units (APUs) can facilitate advanced graphics capabilities within an extremely small footprint, without compromising power and cooling efficiency or cost. The merging of advanced x86 computing capabilities with the parallel processing power of General-Purpose Graphics Processing Units (GPGPUs) in a single device allows OEMs to design low-power, graphics-intensive SFF systems that until now have been exclusive to power-hungry multicore CPUs and add-on graphics cards.

The evolution to increasingly intense graphics

Graphics-driven applications are accelerating the pace of innovation for portable, energy-efficient SFF systems. Applications spanning digital signage, information terminals, point-of-care medical imaging and diagnosis, and industrial applications are evolving to offer advanced graphics performance, but in many cases are constrained by conventional CPU and discrete GPU processor architectures. Here we’ll look at each of these applications individually and address some of their unique design constraints, and also assess the ways in which APUs can minimize these constraints.

Mobile digital signage and information terminals

The travel services industry in particular has embraced digital signage as a means to provide timely, location-aware information. GPS-assisted in-vehicle digital signage and other mobile digital signage better equip travelers for personal use and empower travel services and transportation vendors with “high proximity” advertising space for local businesses. Multi-screen display capabilities are emerging as an important feature for these applications, and mobile digital signage is especially sensitive to power consumption requirements. Low power draw is crucial if a mobile digital sign is to be powered by, for example, a shuttle bus battery.

Point-of-care medical imaging and diagnosis

Portable medical devices with sophisticated medical imaging capabilities for use at the point of care outside of the hospital can enable medical professionals to examine patients in the field, as well as access and process imaging-intensive patient data such as Picture Archiving and Communications Systems (PACS) datasets stored within hospital information systems. These devices ensure high-resolution imaging and ultra-precise diagnostic information that first responders and care providers count on to expedite treatment decisions.

Apart from the inherent design constraints associated with high-performance graphics processing, device portability, and battery-life preservation, medical device designers grapple with stringent device certification processes that often consume valuable time and intense time-to-market pressures that few other industries face as acutely.

Portable industrial applications

Imaging and data-intensive industrial applications such as image detection and recognition, automated inspection, and distributed data collection systems that require high-speed vector processing are increasingly being deployed in remote settings for monitoring purposes, and are therefore sensitive to portability requirements. In addition to requiring increased parallel processing capabilities to facilitate high-precision real-time data collection, these systems often need to be ruggedized for harsh environments. Highly compact, fluid- and particle-sealed system enclosures present obvious challenges to airflow and venting – challenges that are often insurmountable with traditional CPUs due to their thermal profiles.

APUs yield higher performance graphics with fewer components

New-generation boards and modules designed with advanced x86 APUs are ideally suited to minimize and/or eliminate the aforementioned design challenges while maximizing overall graphics performance. The combination of a low-power CPU and a discrete-level GPU into a single embedded APU provides OEMs with optimal picture resolution (frame rates and resolutions of up to 2560 x 1600 pixels, for example) for their graphics-driven, mobile SFF systems. Combining a GPU core on the same die as the CPU enables host systems to offload computation-intensive pixel data processing from the CPU to the GPU. Freed from this task, the CPU can serve I/O requests with much lower latency, thereby dramatically improving real-time graphics processing performance.

Size and integration

APUs also reduce the footprint of a traditional three-chip platform to just two chips – the APU and the companion controller hub. The combination of general purpose CPU and GPU onto a single die with a high-speed bus architecture and shared, low-latency memory model simplifies design complexity through a reduction in board layers and power supply needs, enabling SFF system designers to achieve aggressive form factor goals while driving down overall system costs.

By providing native, high-performance graphics processing at the silicon level, APUs preclude the need for bulky add-on graphics cards that usually require a right-edge connector. In space-constrained designs, an edge connector takes up more space (card-edge boards are typically 3" to 5" taller) and exposes it to additional shock and vibration that can lead to signal integrity issues. Designing APU-caliber graphics capabilities directly onto a carrier board is a more rugged, long-term option.

Power and cooling

The Performance-Per-Watt (PPW) gains enabled by APUs assure greater power efficiency and lower heat dissipation, which in turn can preclude the need for fan cooling within SFF systems, thus helping to preserve board space, improve overall system reliability, limit system noise, and lower BOM costs. Supporting Thermal Design Power (TDP) profiles from 5.5 W to 18 W, with typical power consumption below 6 W[1], AMD G-Series APUs equip designers with the ability to keep board-level total power dissipation to within approximately 35 W, well within the 45 W threshold at which mobile systems begin to become hot and physically uncomfortable to the touch. These factors enable designers to optimize their SFF systems for extremely compact enclosures and/or applications with power constraints, and can help designers stay within the 25 W threshold at which passive cooling is an acceptable (and typically favorable) option.

Multi-display video immersion

The ability to support multiple independent display outputs simultaneously is an emerging requirement for realizing ultra-immersive video displays for digital signage, and also SFF portable medical devices. New-generation APUs enable designers to cost-effectively develop multiple video displays without sacrificing board space for add-on graphics cards and controllers or compromising overall picture resolution. They also offer the ability to decode up to three HD video streams in parallel and support up to four independent digital displays via a wide range of standard interfaces, including DisplayPort, DVI, HDMI, LVDS, and VGA.

Vector processing for SFF industrial systems

Applications requiring increased parallel computing capabilities, such as the portable medical and industrial devices mentioned above, are well suited for boards and modules equipped with APUs. These applications include 3D medical X-ray image reconstruction and smart camera applications such as high-precision image/pattern detection and identification. However, traditional CPU architectures and application programming tools are optimized for scalar data structures and serial algorithms, and as such, are not the best match for data-intensive vector processing applications.

The integration of general-purpose, programmable scalar and vector processor cores for high-speed parallel processing establishes a new level of processing performance for SFF systems at an unprecedented PPW. In the case of AMD G-Series APUs, the general-purpose vector processor cores within the embedded GPU – 80 shader cores running at 500 MHz (AMD Fusion T56N) – drive the ultra-high-speed processing required to handle intensive numerical computations.

Time to market

The inherent architectural advantages introduced with APUs go a long way toward minimizing design complexity and accelerating time to market. These advantages are owed primarily to reductions in board layers, discrete add-on processors/cards, and power supply and cooling needs, which naturally minimize the number of components on the board and therefore enable designers to shorten, and in some cases eliminate, design cycles.

The underlying x86 APU architecture also enables portable SFF system designers to tap into the vast selection of existing x86-optimized software, applications, and development environments available on the market, introducing additional opportunities to enhance development efficiency and speed time to market. The open development ecosystem for the AMD G-Series platform, for example, includes support for Linux, Microsoft Windows, and Real-Time Operating Systems (RTOSs), multiple BIOS options, OpenGL 4.0 and OpenCL support, and source-level debug tools.

By implementing AMD G-Series APUs on the most common form factors for graphics-intensive applications, such as Computers-On-Module (COMs) and SFF SBCs and motherboards, Kontron is making the benefits of this new x86 processing architecture readily available for application development. OEMs and system integrators can take advantage of highly scalable, validated APU-based platforms that streamline design cycles and minimize design risks to ensure fast time to market for graphics-intensive and parallel-data SFF applications.

Making graphics performance goals achievable

New APU processor architectures are making a fast and transformative impact on SFF design initiatives, unlocking high-performance graphics capabilities in small form factors that simply can’t be achieved with conventional CPUs and GPUs. Continued innovation in the APU domain promises to push graphics performance boundaries even further, and will ultimately yield a new generation of portable SFF systems that defy space, power, and cooling limitations in ways previously unimagined.

Kelly Gillilan is the Product Marketing Manager for the AMD Embedded Solution division, overseeing worldwide marketing strategy and activities. He has worked extensively in embedded applications for most of the past decade. Kelly holds a degree in Computer Engineering and is fluent in Mandarin Chinese.

Christine Van De Graaf is the Product Manager for Kontron America’s Embedded Modules and Small Form Factor SBCs product families. Christine has more than a decade of experience working in the embedded computing technology industry, and holds an MBA in marketing management from California State University, East Bay.

AMD kelly.gillilan@amd.com www.amd.com

Kontron christine.vandegraaf@us.kontron.com www.kontron.com

References

[1] For complete test and configuration information please refer to the AMD “Brazos” Platform Performance and Power Optimization Guide Publication #48109 Rev 2.01 available on the AMD Embedded Developers Support Web site.

(click graphic to zoom by 1.9x)

SFF: Portable medical devices have come a long way in the last few years. What impresses you about the current state of technology in these types of devices?

TABORN: The cell phone market, among others, is driving cost, size, power, and ease of use improvements and possibilities into all application areas, which in medical modalities are quickly being implemented to improve patient outcomes. The handheld ultrasound and its battery life is a great example. The most impressive impact on medical devices is the better focus on user experience. This will directly improve patient care by decreasing the error rate of the applications and evaluation of the data.

McCRACKEN: Out of nowhere, Android has emerged with the potential to become a dominant platform for portable embedded computing devices in the not-so-distant future. Chip-scale integration and improvements in battery technologies accompany the demand for standardized software application platforms. Finally, the performance of ARM SoCs has increased (up to Gigahertz dual core), while Intel has lowered its entry-level ultra-mobile processors to fit within size and power envelopes in order to compete for these coveted high-volume applications.

CHUNG: The projective capacitive multi-touch screen has become one of the hottest topics within this segment, but requires application software development to showcase its values. Also, energy efficiency has always been a key focal point for portable devices, and the rise of RISC-based solutions has helped to further energy savings.

Additionally, the different types of connectivity including Wi-Fi, Bluetooth, and 3.5G/4G wireless that are now readily built into portable medical devices permit easy access of electronic medical record databases or the future medical cloud in any location equipped with wireless signal reception.

MUNCH: The acceptance of x86 and Windows into a market that has traditionally relied on custom hardware and software solutions is impressive. We see Windows as the primary user interface tied to FPGAs performing data crunching in many medical applications. There is also an increasing desire to use standard building-block products like COM Express CPU modules to allow the product design to be focused on its core competency, which is increasingly software.

SFF: What design challenges are engineers currently facing in medical device development?

McCRACKEN: One key task facing engineers is platform selection. Everything from design environment and development tools to production royalties to product updating in the field hangs in the balance. Android is optimized for ARM at the moment, while other Linux platforms and Windows Embedded Compact run well on ARM and x86/Intel architectures alike. Additionally, time-to-market pressures are becoming as critical for FDA and other regulatory-based markets as they are for commercial and consumer markets where the winners take all. To that end, the richness and completeness of a product offering’s “out-of-the-box” functionality translates directly to competitive advantage. SBCs and COMs need to be ready as close as possible to the silicon launch (mass production).

MUNCH: There has been a significant increase in the speed of signals in today’s designs, resulting in the need to use expensive and complicated simulation tools to verify signal integrity. Waiting until a design is fabricated to check and catch signal integrity issues can impact launch schedules and development costs. Even when using module building blocks the design still needs to deal with high-speed interfaces such as PCI Express, SATA, and now USB 3.0 SuperSpeed.

TABORN: Of the many challenges engineers face, designers must first consider security since virtually all medical devices in the future will be connected to some type of network. Second, there are new demands for “ease of use” that are fostered by what many in the industry call the “iPhone factor.”

CHUNG: Medical customers see features like low cost, long battery life, light weight, slim design, and new technologies that are currently seen in consumer products, and expect to see these elements implemented in portable medical products, but medical devices do not yet have these features.

SFF: Where do you expect medical devices to go in the future?

CHUNG: Eventually portable medical devices will be used in the same sense that we use smartphones and tablets in our daily lives, but within a more secure network and with mechanisms to permit/deny access to sensitive patient data. Not only the patients but the physicians and healthcare administrators will benefit significantly from this development.

From a hardware perspective, lighter and thinner is always the trend. On a system level, portable medical devices have different market segments, such as general hospital/clinical administration usage that may require building a whole infrastructure, or portable diagnostic devices for ultrasound, ECG, and blood pressure monitoring that require joint system design with customers.

McCRACKEN: Someday, the portable subset of embedded devices will be nearly as ubiquitous as their consumer counterparts, relatively speaking. Whether in the form of sensors or medical patient monitors, these products will proliferate based upon consolidated, standardized ultra-mobile platforms much the way the original DOS + x86 embedded computers did. In some cases with dual- or multicore systems, the second processor core will be devoted to the deterministic and real-time aspects of the device, such as taking measurements.

TABORN: These devices will be far more flexible and extensible in the future. One of the best things to happen to medical is the advent of the iPhone. This demonstrated to the world that a small device could be intuitive and very efficient. This will cause device manufactures to address the areas of ease of use and human workflow, reducing human error and encouraging operation and use cases in non-traditional settings – improving patient care throughout the world.

SFF: What does the industry need to get to the next generation of medical portable mobile devices?

MUNCH: Continuing to drive down total power consumption would be a good start, and achieving this will just take time. This results in two benefits: an increase in battery life (or a smaller battery to reduce weight) and reduction in power that needs to be dissipated.

CHUNG: The prospect of wireless battery charging would be one technology that would help these applications realize their true potential for power-efficiency and application usage.

Additionally, comprehensive infrastructure, regulations, and mobile healthcare protocols will be key to these devices’ future. This will allow medical computing manufacturers the ability to develop portable/mobile medical-specific devices and applications that can be implemented within the same network, making the future medical cloud ecosystem possible.

TABORN: The ability to implement future security policies must be considered in today’s devices. This suggests having the “headroom” in the design shipped today to be able to implement more complex policies in the future. Unlike most devices, medical devices in clinical and hospital settings are unique in that they can be in service for 10-to-15 years. For consumer-geared medical mobile devices, we will have to ensure that the applications data acquired can be just as safe and reliable as data acquired in the clinical setting (given the various different circumstances). Multicore solutions are becoming readily available in both Intel and ARM architecture families. This topology choice will allow developers to address these unique application requirements for today with the necessary performance headroom to support the ever-changing security landscape.

McCRACKEN: Better hardware standards are needed in order to “cross the chasm.” Some existing standards like Qseven have been reasonably architecture-independent. However, there are now so many single-vendor-driven x86 and (especially) ARM module and interface standards that have been prevented from reaching critical mass in this industry. A casual stroll down the halls of Embedded World in Germany reveals that a massive shake-out will be needed; otherwise system manufacturers will be left squandering time-to-market and development budgets in taking the full custom path. Standards organizations have been portrayed as slow-moving and political, leading some suppliers to go it alone. Any standards groups that can set aside self-interests and become more responsive to customers and end users will have a major leg up in leading the consolidation that is needed to facilitate the next wave of medical portable devices.

Maximizing performance is especially important for military embedded systems because of the growing need to keep costs low while satisfying the requirements of connectivity in an increasingly digital battlefield. As manufacturers reach the limits of what can be wrung from increased miniaturization and integration, the best approach to increased performance is the use of multicore processors.

The downside is that to take full advantage of many cores executing in parallel, the software must be written to be intrinsically multithreaded. Software written to be single-threaded for a single core processor will realize little or no performance benefit when executed on a multicore processor: It must be rewritten or adapted to use multithreading. The key challenge is to keep the cores busy as much as possible, while ensuring that they coordinate access to shared resources properly. Unfortunately writing such code is much harder than writing single-threaded code. When there are defects such as deadlocks or race conditions, they can manifest in ways that are difficult to diagnose. Traditional techniques for finding and eliminating concurrency bugs may be ineffective.

One of the core reasons why concurrency bugs are so difficult is because there is an enormous number of ways in which the events in the threads can be interleaved when those threads execute. As the number of threads or instructions increases, the number of interleavings increases exponentially. If thread A executes M instructions and thread B executes N instructions, there are N+MCN possible interleavings of the two threads. For example, given two trivial threads with 10 instructions each, there are 184,756 possible interleavings of those instructions. Even with very small programs it is clear that it is next to impossible to test all possible combinations. Secondly, even if it is possible to identify a single interleaving that leads to a failure, it can be very difficult to set up a repeatable test case that uses that particular interleaving because scheduling of threads is effectively nondeterministic. Consequently, debugging concurrent programs can be very expensive and time consuming. A race condition is a class of concurrency defect that is easy to accidentally introduce and difficult to eliminate with conventional testing. However, there are techniques programmers can use to find and remove them.

Potential catastrophic failures

Compared to single-threaded code, entirely new classes of defect can occur in concurrent programs, including deadlock, starvation, and race conditions. Such defects mostly cause mysterious failures during development that are very difficult to diagnose and eliminate. One avionics manufacturer we have worked with spent two person-years applying traditional debugging techniques in an effort to find the root cause of an intermittent software failure that turned out to be a race condition. Sometimes the consequences can be dire – two of the most infamous software failures ever were caused by race conditions. The Therac-25 radiation therapy machine featured a race condition that was responsible for the deaths of several patients[2]. Similarly, the 2003 Northeast blackout was exacerbated by a race condition that resulted in misleading information being communicated to the technicians[3].

There are several different kinds of race conditions. One of the most common and insidious forms – data races – is the class of race conditions involving access to memory locations.

A data race occurs when there are two or more threads of execution that access a shared memory location, at least one thread is changing the data at that location, and there is no explicit mechanism for coordinating access. If a data race occurs it can leave the program in an inconsistent state.

Consider avionics code that controls the position of a flap. In normal circumstances the flap is in a position dictated by the flight control software, but the pilot can override that position by pressing a button on his control panel, in which case a manually set position is used. To keep things simple, let’s say that there are two threads in the program: one that controls the flap and one that monitors the position of the elements on the control panel. There is also a shared Boolean variable, named is_manual, that encodes whether the manual override is set or not. The flap position thread checks the value of is_manual, and if true, it sets the position accordingly. The control panel thread listens for button press events, and if the override button is pressed, it sets is_manual to true. Figure 1 shows the code that one might write to implement this specification. This code is likely to work most of the time; however, because the is_manual variable encodes a state that is shared by both threads, it is vulnerable to a data race because access to it is not protected by a lock. If the flap positioning code is being executed at the exact time that the pilot hits the override button, then the program may enter an inconsistent state and the wrong flap position will be used. Figure 2 shows how this might happen.

(click graphic to zoom by 1.9x)

(click graphic to zoom by 1.9x)

This example neatly illustrates one of the properties of data races that makes them hard to diagnose: The symptom of corruption may only be observable long after the data race has occurred. In this case, the fact that the wrong flap position is being used may only be noticed when the pilot notices the aircraft is not responding as expected.

A widely held belief is that some instances of data races are benign and can be tolerated. However, it is now clear beyond doubt that this is only rarely true. The C standard[4] states unambiguously that compilers are allowed to assume that there are no data races, so optimizers can and do make transformations that are valid for improving the performance of single-threaded code but which introduce bugs when there are apparently benign race conditions. These are subtle effects – even experienced programmers are regularly surprised by them. (See reference [1] for a full explanation and several compelling examples.) Because of this, to achieve high levels of assurance and avoid disastrous failures, it is very important to find and remove all data races.

Eliminating concurrency defects

Given that concurrency defects, and data races in particular, are so risky, it is important to use multiple techniques to eliminate them. Traditional dynamic testing is not well suited for finding many concurrency defects because of non-determinism. A program that passes a test hundreds of times may later fail in the same environment with exactly the same inputs because the bug can be exquisitely sensitive to timing. Engineers looking for high assurance must turn to other techniques if they are to eliminate concurrency defects.

Static analysis tools offer a means for finding such bugs. The key difference between testing and static analysis is that it tests a particular execution of a program for a given set of inputs, whereas static analysis finds properties that are good for all possible executions and all inputs. (In practice, static analysis tools make approximations to achieve acceptable performance and precision, so fall short of this ideal model. Nevertheless, they do cover many more cases than would ever be possible with traditional testing.)

Roughly speaking, static analysis tools work by creating a model of the program and by doing a symbolic execution of that model, looking for error conditions along the way. For example, GrammaTech’s CodeSonar static analysis tool finds data races by creating a map of which locks are held by which threads and by reasoning about the possible interleavings that could result in unsynchronized access to shared variables. Deadlock and other concurrency defects (including lock mismanagement) are found using similar techniques.

Custom concurrency constructs: A case study

Standard defect detection techniques are most useful when programs use standard ways of managing concurrency. Most tools recognize and can reason about the special properties of standard libraries such as the POSIX threads library or proprietary interfaces such as VxWorks. However, many systems use custom techniques for managing concurrency.

For example, another manufacturer we worked with built a safety-critical device on a platform that used a custom pre-emptive multithreaded software interface. In this design, a key constraint was that all data instances that could be accessed from multiple priority levels of threads had to be protected with proper guard constructs. Prior to using static analysis, validating that this constraint was respected required a person-month of manual analysis. To reduce the cost, they sought a solution by turning to static analysis. An important property of modern advanced static analysis tools is that they are extensible: They provide an API with abstractions that make it convenient to implement custom static-analysis algorithms. Using CodeSonar’s API, they were able to program a solution that piggybacked on the algorithms used at the core of the existing analyses to find locations in the code where the design constraint was being violated. The resulting tool, implemented as a plug-in, is able to find violations of the key constraint automatically, all at a fraction of the cost and in much less time than was previously possible.

Multicore trade-off

There are compelling reasons to move to multicore processor designs, but the risk is that doing so introduces the possibility of concurrency defects in the software. These are easy to introduce – even apparently innocent code can harbor nasty multithreading bugs – and notoriously difficult to diagnose and eliminate when they occur. Traditional testing techniques alone are inadequate to ensure high-quality software, mainly because of the high degree of nondeterminism. The use of advanced static analysis tools that use symbolic execution is one approach that can help because such tools can reason about all possible ways in which the code can execute. These tools can find defects such as data races and deadlocks in code that uses standard multithreading libraries, and can even be adapted to designs that use nonstandard concurrency constructs.

References

[1] Boehm, H.-J., How to miscompile programs with “benign” data races. In HotPar’11 Proceedings of the 3rd USENIX conference on Hot topics in parallelism.

[2] Leveson, N.G., An investigation of the Therac-25 accidents. IEEE Computer, 1993. 26: pp. 18-41.

[3] Poulsen, K., Tracking the blackout bug, www.securityfocus.com/news/8412.

[4] C Standards Committee (WG14). Committee Draft: www.open-std.org/jtc1/sc22/wg14/www/docs/n1539.pdf

Paul Anderson is VP of Engineering at GrammaTech. He received his B.Sc. from Kings College, University of London and his Ph.D. in Computer Science from City University London. Paul manages GrammaTech’s engineering team and is the architect of the company’s static analysis tools. Paul has worked in the software industry for 20 years, with most of his experience focused on developing static analysis, automated testing, and program transformation tools. Contact him at paul@grammatech.com.

GrammaTech, Inc. 607-273-7340 www.grammatech.com

Programming multicore processors to take advantage of their power means writing multithreaded code. C and C++ were not designed for concurrency, so developers must use a library such as pthreads for those languages. Multithreaded code is more difficult to get right than single-threaded due to the risk posed by entirely new classes of programming defects.

In the rogue’s gallery of concurrency bugs, the race condition is a notorious repeat offender. A race condition occurs when a program checks a resource property and performs an action on the assumption that the property has not changed, even though an external actor has slipped in and changed that property.

A data race is a particular type of race condition that involves concurrent access to memory locations in a multithreaded program. This defect occurs when there are two or more execution threads that access a shared memory location, at least one thread is changing the data at that location, and there is no explicit mechanism for coordinating access. If a data race occurs, it can leave the program in an inconsistent state.

The insidious nature of data races

It is widely assumed that some data races are harmless and can be safely ignored. Unfortunately, this is only true in very rare circumstances. It is best to explain why by introducing an example.

The singleton pattern is a commonplace idiom where the program maintains a reference to a single underlying object and a Boolean variable encodes it if it has been initialized. This pattern is also known as lazy initialization. The following code is an example of the pattern:

if (!initialized) {

object = create();

       initialized = true;

}

... object ...

This code is perfectly adequate for a single-threaded program, but it is not thread-safe because it has a data race on the variable named initialized. If called by two different threads, there is a risk that both threads will observe initialized as false at essentially the same time, and both will call create(), thus violating the singleton property.

To make this thread safe, the natural approach is to protect the entire if statement with a lock. However, acquiring and releasing locks can be costly, so programmers try to avoid the expense by using the double-checked locking idiom – a check outside the scope of the lock and another inside. The inner check is there to confirm that the first check still holds after the lock has been acquired:

if (!initialized) {

       lock();

       if (!initialized) {

            object = create();

            initialized = true;

       }

       unlock();

}

  ... object ...

Superficially, this looks like it will suffice, and indeed, it will as long as the statements are guaranteed to execute in that order. However, an optimizing compiler might generate code that essentially switches the order of object = create() and initialized = true. After all, there is no explicit dependence between those two statements. In that case, if a second thread enters this code any time after the assignment to initialized, that thread would then use the value of object before it has been initialized.

Optimizing compilers are inscrutable beasts. Those that optimize for speed will take many esoteric considerations into account, few of which are obvious to a programmer. It is common for them to generate instructions that are apparently out of order because doing so might result in fewer cache misses, or because fewer instructions are needed.

It is wrong to assume that because the reordering introduced a race condition in the previous example that the compiler is at fault. The compiler is doing exactly what it is allowed to do. The language specification is perfectly clear and unambiguous about this: The compiler is allowed to assume that there are no data races in the program.

In actuality, the specification is somewhat broader: Compilers are allowed to do anything in the presence of undefined behavior. This is sometimes facetiously referred to as catch fire semantics; the specification gives the compiler permission to set a computer on fire if the program has undefined behavior. As well as data races, many traditional bugs such as buffer overruns, dereferences of invalid addresses, and so on constitute undefined behavior. Because compilers are free to do anything, rather than burn the building down they typically do the sensible thing, which is to assume that the undefined behavior will never happen and optimize accordingly.

The consequences of this can sometimes be surprising, even to those who are experts in concurrency and compilers. It can be difficult to convince programmers that code that looks completely correct can be compiled into code that has serious errors.

Another example is worth describing. Suppose there are two threads where one reads a shared variable and the other writes to it. Let’s assume that it does not matter to the reader if it sees the value before or after it has been changed by the writer (this is not an uncommon pattern). If those accesses are not protected by a lock, then there is clearly a data race. Notwithstanding the catch fire rule, however, most programmers would conclude that this is completely benign.

As it turns out, there are at least two plausible ways in which this code could be compiled where the reader would see a value that is wrong. The first way is easy to explain: Suppose the value were a 64-bit quantity on an architecture that can only read 32-bit words. Then both the reader and the writer need two instructions, and an unlucky interleaving might mean that the reader sees the top 32 bits of the old value and the bottom 32 bits of the new, which when combined can be a value that is neither the old nor the new.

The second way in which wrong code could be generated is more subtle. Suppose the reader did the following, where the data race is on the variable named global:

int local = global;  // Take a copy of
                       // the global

if (local == something) {

       ...

}

... // Some non-trivial code that does
      // not change global or local

if (local == something) {

       ...

}

Here the reader is making a local copy of the racy variable and referring to that value twice. It is reasonable to expect that the value will be the same in both places, but again, the optimizing compiler can generate code where that expectation is unmet. If local is assigned to a register then it will have one value for the purposes of the first comparison, but if the code between the two conditionals is sufficiently nontrivial then that register may get spilled – in other words, reused for a different purpose. In that case, at the second conditional, the value of local will be reloaded from the global variable into the register, by which time the writer might have changed it to a different value.

Programmers should be very skeptical of claims that some data races are acceptable and should strive to find and remove all of them from their code.

Techniques for finding risky defects

When it comes to finding concurrency defects, traditional dynamic testing techniques might be inadequate. A program that passes a test a hundred times is not guaranteed to pass the next time, even with the same inputs and the same environment. Whether these bugs manifest or not is exquisitely sensitive to the timing, and the order in which operations in threads are interleaved is essentially nondeterministic.

New dynamic testing techniques for finding data races are emerging. These techniques work by monitoring the applications as they execute and observing the locks held by each thread, as well as the memory locations being accessed by those threads. If an anomaly is found, then a diagnostic is issued. Other tools help diagnose data races suspected of causing failures. Some companies now offer tools to facilitate diagnosis of data races that allow the replay of events leading up to an anomaly.

Static analysis tools can also be useful for finding data races and other concurrency errors. Whereas dynamic testing tools find defects that occur for particular executions of a program with a fixed set of inputs, static analysis tools check all possible executions and all possible inputs. For performance reasons, tools might place limits on how much exploration is done and thus might not be completely exhaustive; even so, they can cover much more than can ever be feasible with dynamic testing. The advantage of static analysis is that test cases are not required because the program is never actually executed.

Instead, these tools work by creating a model of the program and then exploring the model in various ways to find anomalies. GrammaTech’s CodeSonar finds data races by creating a model that represents the set of locks held by each thread and by performing a symbolic execution of the program that explores execution paths. It records the sets of variables protected by locks and uses this information to find interleavings that can result in shared variables being used without proper synchronization. Similar techniques can be used to find other concurrency defects such as deadlock and lock mismanagement.

Once found, data races are usually easy to fix, albeit doing so correctly can incur a performance penalty. In some cases, there might be a temptation to use the C volatile keyword to correct the data race, but this is not recommended because volatile was not designed to solve concurrency problems, and in any case is a poorly understood construct that is frequently miscompiled. The latest versions of C and C++ have embraced concurrency and support atomic operations. Compiler support for these operations is slowly emerging, and until it becomes readily available, the best approach is to use locks.

To achieve high-quality software for multicore processors, a zero-tolerance policy for data races is recommended. Find them using a combination of both static and dynamic techniques, and take care not to rely too heavily on esoteric compiler techniques to fix them. These defects are so risky and unpredictable that eliminating them systematically is the only safe way to be sure that they do not cause harm.

Paul Anderson is VP of Engineering at GrammaTech.

GrammaTech
607-273-7340
paul@grammatech.com
www.grammatech.com

ECD: As embedded developers turn to multicore processors to optimize performance, how can analysis tools help control inevitable cost and schedule problems?

FISHER: Any new development is an exercise in balancing expectation against risk. In the case of multicore, the naive expectation is always linear acceleration tempered perhaps by some jocular “wouldn’t that be nice” acceptance that the final result won’t be quite that good, but no real understanding of the reality that without significant effort (read: time, money, angst) the result might be slower than the old, interrupt-driven single-core code. So tools have a role to play in terms of helping developers understand the impact of what they’re doing, what pitfalls they’re unwittingly leaving themselves open to, and how to mitigate the associated risks.

Dynamic analysis in this space has received the lion’s share of attention for obvious reasons. If I can see a nice set of graphs over time that shows the performance of my code in action, I can presumably narrow in on problem areas quickly and apply my own knowledge to figuring out what’s going on. The challenge with dynamic analysis is that it depends on a) defining a test set that shows execution problems, and b) the reviewer’s intimate knowledge and understanding of what to do about those problems.

Static analysis, by contrast, assumes little knowledge on behalf of the reviewer and requires no effort to be expended in defining test cases. Every conceivable code path through the application is exercised with as much rigor as every other path. This approach is therefore far more likely to show complex and costly issues such as data races, deadlocks, and resource contention than any constructed test bench. That speaks directly to the bottom line of cost control in what is inevitably a large, over-budget project. Leaving issues such as these in a code base until late in the validation process will cost exponentially more to address than doing so during initial development.

In addition, due to the way that static analysis works by modeling the expected program execution, the finding is accompanied by a detailed walk-through of how the situation is predicted to occur, allowing even a relatively junior resource to interpret the issue at hand, determine whether or not it’s likely to happen, and apply an appropriate design fix. In one example we like to describe in seminars on the subject, a design flaw in a popular open-source database kernel resulted in months of effort expended to identify a deadlock and eventually rewrite key modules to avoid the data race at its heart. This same problem was identified during the first analysis using our tools, which provided a walk-through description enabling developers to easily see that the data race was causing the problem and that the deadlock was merely the symptom.

Contrast a few hours to run the tool to analyze the code and an hour at most to interpret and act upon the result (what turns out to be a one-line fix) with months of community effort to determine an appropriate set of tests, followed by design effort attempting to fix the deadlock, and finally requiring the designer to rewrite the whole thing from scratch.

ECD: How is static analysis introduced in the software development cycle, and how can it be used with existing Integrated Development Environment (IDE) tools?

FISHER: There’s absolutely no doubt that any developer-facing tool that doesn’t integrate seamlessly with any existing tooling is going to face significant friction in deployment. We’ve been selling this message effectively for years, with development managers almost asking the “why wouldn’t you do it this way?” question for us.

Whether developers have migrated to IDEs or their idea of an IDE is a bunch of gVim macros or emacs lisp modules, to them it’s where they live and work. And woe betide any vendor who tries to get them to change. Even if you’re not suggesting that they change tools, and instead asking them to visit somewhere else to see what they might have done wrong in some retrospective manner, your tool is going to suffer waves of disinterest and ultimately become shelfware.

Thus, static analysis has to be part of the developer’s native habitat, and more importantly, it has to work in a way that feels natural and follows the way other tools in that habitat work. For a gVim developer, issuing a ‘:’ command is second nature, so tools in that environment should follow suit. Put that same interaction mechanism in front of a Visual Studio user, and that will make for a fun-filled afternoon of derisive commentary.

At Klocwork we’ve gone through various iterations of technology design, getting closer to the developers themselves. It’s one thing to be resident as a tool within an IDE; it’s another to get the developer to “push the button” to use it. Making a button available is only changing geography, and it does nothing to help with the tool’s fundamental usability.

With this in mind, we’ve recently introduced a new technology that allows static analysis to take place in much the same way as spell checking in a Word document or e-mail. That is, as you’re writing your code and the tool detects a problem with what you’re doing, it can point out the problem in a perceptually instant manner, highlight it with a squiggly underline, and deliver all the value of static analysis with none of the “what do I have to do to use it?” resistance that is typically encountered during any new tool’s introduction (see Figure 1).

(click graphic to zoom by 1.9x)

Getting a tool into an IDE is tough; getting it to be useful in the part of the IDE the developer truly lives in (the editor component, whatever that looks like in the environment at hand) is really tough. But until you’re there, you’re a distraction.

ECD: Can source code analysis be used to protect embedded devices against potential security threats?

FISHER: Absolutely, source code analysis has a significant role to play in threat identification and validating whatever threat model is being used to determine vulnerability in the device. The connectivity requirement is common in the embedded world today. That connection might be to another chip, or another device, or the whole Internet. In any case, there’s somebody else either sending you information or receiving information you’re sending. That’s the starting point for having to worry about your entire application design.

Static analysis doesn’t typically know, or try to know, anything about your surrounding environment. Tools can sometimes be tuned to perform their analysis within certain data boundaries, for example, such as knowing that a particular input will only range between -20 and +30 because it’s a temperature sensor intended for use in Western Europe. But that kind of thing is discouraged because you’re allowing the user to define limits to what the modeling technology naturally does – that is, assume nothing and point out everything that looks wrong.

In the case of threat detection, we’re most worried about how the data you’re interacting with from the outside world is used internally. Is it used to create a buffer into which you’ll read data (code or value injection), or is it used for memory allocation (Denial of Service or DoS), or perhaps interpreted as a reference into an internal data structure (hijacking or redirection)? This kind of data and path validation – that is, the path that tainted data follows from the outside world to its point of use within the code – is natural for source code analysis, as it accomplishes this modeling in order to perform everything else it does.

As a wonderful gentleman at a defense contractor once said to me, “Son, it’s a bomb; it’s supposed to blow up. We just want to be sure it doesn’t get hijacked on its way.”

ECD: What educational events or online classes does Klocwork offer to help embedded designers get started with its code analysis tools?

FISHER: Like any commercial organization attempting to encourage users to gain value from its tools, Klocwork provides a full suite of educational and professional services, running the gamut from introductory materials aimed at first-time users, to more advanced courses targeting secure coding and threat modeling, to full-on deployment services and mentoring.

We also recently introduced the Klocwork Developer Network (http://developer.klocwork.com), a website serving our users and acting as a repository for online courses, video tutorials, in-depth courseware, and the usual variety of community forums and ticketing.

Each customer has something unique they wish to gain from a tool such as source code analysis, so a large part of our educational focus is on internal champions, people who take knowledge of how the tools work and how other organizations have applied them and leverage those lessons in deploying our tools for their own use. A large part of that is learning from the community, so I’ve been thrilled to see the fast uptake that the Klocwork Developer Network has seen amongst users, both as a less formal mechanism for interacting with our staff, but most importantly as a way to learn from other customers.

(click graphic to zoom)

Gwyn Fisher is CTO of Klocwork.

Klocwork info@klocwork.com www.klocwork.com/blog www.facebook.com/klocwork www.twitter.com/@klocwork www.klocwork.com

Some believe that the Mayan calendar forecasts the end of the world in 2012, but this year promises to have much adventure for VITA technologies. Here are
10 trends to watch unfold in 2012:

1. VPX revenues are projected by industry analysts to match VME in 2012. The numbers are rapidly approaching the crossover. New product announcements featuring VPX are in the majority. Not all systems need the performance of VPX so VME design wins continue, but design wins featuring 3U and 6U VPX products are becoming more common, raising expectations that this is the year.

2. Military programs will take a hit in 2012. Upgrades will be the safe harbor, but they don’t offer much opportunity for companies looking to enter the market, as these opportunities usually go to the incumbent supplier. At the same time, defense programs look to reduce manpower costs by using more automation and robotics, especially in unmanned vehicles of all types. 2012 promises to be a tough year but could offer great opportunities for companies positioned well with products and business strategies to address this automation.

3. Small form factor fever rages on in 2012. Making products smaller while not giving in on capability and performance promises to open up new markets. VITA has four small form factor working groups in play. Which will win is yet to be determined, but we may get a clearer picture by the end of the year. Count 3U VPX in this mix as well.

4. FPGAs are poised to take over the majority of I/O responsibility on single board computers. They offer a cost/flexibility ratio never before possible, and it only promises to get better. FMC technology makes the use of FPGAs very attractive as board supplies look for ways to create niches for their expertise.

5. Intel just bought the InfiniBand product lines and certain related assets from QLogic. InfiniBand dominates the storage connectivity markets, but does this acquisition mean a more prominent role for InfiniBand in critical embedded systems? Will we start to see the dot specifications for VPX and other VITA technologies fill in the InifinBand gaps? 10 Gigabit Ethernet is not going to give any ground any time soon.

6. Optical interconnects have been discussed for decades, but technology breakthroughs combined with eventual approach to copper bandwidth limitations make even more critical the search for optical interconnects that are cost effective and practical. 2012 promises to show some of the first VPX products at least allowing optics to be passed through the backplane using VITA 66 blind mate optical interconnect.

7. With all the focus on small form factors, will we see proposals for smaller mezzanines for blade boards appear in a working group? The technical community has been discussing options, but none of them seem to ever gain any traction. 2012 could be the year we see some serious cards get played.

8. PowerPC processors held the dominant spot on VME single board computers for many years. But Intel architecture processors have since taken over the top spot. Can the mistakes made with Power Architecture be corrected in time to regain some of that market share? QorIQ with AltiVec makes a return in 2012. Keep an eye on the progress.

9. ARM processors are everywhere. With more than 200 licensees, it is impossible to find a processor supplier who does not offer an ARM option. Multicore offerings start to put ARM near the high end of performance. Many I/O boards use some type of ARM processor. Pair these with the low power capability and the multitude of processor options and it becomes only a matter of time before ARM shows up as the primary processor in single board computers.

10. Reliability is a key part of the definition of “critical embedded systems.” The work done by the Reliability Community has received high marks within the DoD. MIL-HDBK-217 needs updating, and VITA 57 can help. Will the suppliers start to use the guidance of VITA 57 to define the reliability levels of their products? Many in the user community would like to see that happen in 2012.

To keep pace with the explosion of traffic in the enterprise and carrier network, embedded designers have tried a variety of methods to meet the demand for 100G secure communication, including embedding hardware accelerators into multicore processors or using devices such as network processors, Ethernet switches, or Ethernet controllers. These approaches each come with their own drawbacks that limit performance and increase complexity. Furthermore, attempts to use a single-chip heterogeneous multicore processor to bypass performance issues have led to proprietary architectures that are not operating system friendly.

A high-performance multicore heterogeneous architecture builds on a single-chip multicore heterogeneous processor, but divides the solution into two processors: a general-purpose multicore x86 CPU focused on application and control plane processing and a separate in-line multicore coprocessor focused on L2-L4 processing and accelerating L4-L7 applications. The key to this architecture is having a tightly coupled interface between the two processors that is in-line, secure, virtualized, and high-performance (see Figure 1). A good analogy here is the use of a graphics processor unit alongside a multicore x86 processor in workstations and other graphics-intensive servers.

(click graphic to zoom)

Efficient processing and memory utilization

The coprocessor needs to access packets, forwarding the packet and its associated metadata (packet state) in a timely manner with minimal latencies. This dictates having hierarchical memory architecture of on- and off-chip memories, with packet data effectively managed through the hierarchy. For example, first-level lookup tables can be in on-chip memories, while large volumes of data can be stored in external memory tables.

In addition, the use of multiple threads per processing core can bypass the memory wall problem. A core or thread continues to execute until an external memory access is needed, at which point another processing thread takes over. The resulting asynchronous memory architecture decouples external memory accesses from processing, maximizing overall system performance. This allows for bulk memory transactions, where many memory accesses are pooled together into one memory transaction, further increasing the efficiency of the external memory interface.

In-line or look-aside processing with security and virtualization

The coprocessor is in-line with ingress and egress traffic and should be able to, on-the-fly, encrypt and decrypt the packets, classify packets into flows, and look up the flow state table to determine the action needed on the flow. The coprocessor also implements I/O virtualization, allowing the x86 cores and their Virtual Machines (VMs) to share the I/O subsystem. In addition, the coprocessor should be able to dynamically load balance the traffic to the x86 cores and VMs based on flows.

Fast interconnect with x86

Supporting a heterogeneous processing architecture requires a high-performance interconnect to the x86 processor with I/O virtualization capability. For example, an 8-lane PCI Express Gen 2 interface supports up to 40 Gbaud of traffic to an x86 CPU socket. (Note: Overhead on read and write cycles brings this number down to the low 20s.)

Cut through/intra-flow cut through

Ideally, not all flows need to be transmitted to the x86 processor, as the coprocessor is intelligent enough to classify packets into flows. Based on the flow state table, an action can be taken to cut through, drop, or forward to x86. In some cases, the first few packets of a flow are forwarded to the x86 subsystem for inspection. The x86 processor can then instruct the underlying coprocessor to cut through the remainder of the packets in the same flow.

Inter-VM switching

The advent of VMs and the need for I/O virtualization have created a new set of requirements that mandates a more intelligent approach for managing I/O. This has prompted the need for an intelligent way to interconnect VMs on different cores to handle the so-called east-west traffic. Such VMs can belong to different tiers of servers in the data center. Having the VM-aware switch on the coprocessor can achieve the VM interconnect.

Passive NIC mode

The coprocessor should be able to support a mode where all network I/O traffic is passed to the x86 processor. This mode is required for monitoring and statistics or for applications requiring 100 percent of x86 CPU processing.

Implementing the OpenFlow protocol

Software-Defined Networking (SDN) allows users to bring the benefits of virtualization – including shared resources, user customization, and fast adaptation – to the switched network by defining traffic flows and deciding how these flows are treated in the network. In other words, it allows the system user to remotely control the network hardware with software in a dynamic and programmable fashion.

SDN puts the intelligence of the network into a hierarchy of controllers. In this hierarchy, switching paths are centrally calculated based on IT-defined parameters and then downloaded to the distributed switching architecture. A hardware-agnostic architecture that uses standard open interfaces to the hardware can change the way we build networking systems today.

The new OpenFlow protocol supports SDN. An OpenFlow controller typically runs on a multicore x86 processor and implements the control plane protocols. It downloads the state information onto multiple flow state tables in the coprocessor, implementing the data-switching plane through a standard OpenFlow API. The coprocessor, being a stateful flow processor, can be optimized to support the OpenFlow architecture.

Best of breeds for the future

As line speeds continue to grow, it remains to be seen how application workloads will be divided among x86 general-purpose multicore CPUs and external supporting coprocessors. The flexibility of riding a product roadmap for multicore x86 processors separate from that of coprocessors gives designers the choice to use the best of breeds in trying to meet ever-increasing future challenges.

Editor’s note: Read Part 1 in this series online at http://embedded-computing.com/managing-processors-40100g-part-of-2.

Nabil G. Damouny is senior director of strategic marketing at Netronome.

Netronome 408-496-0022 info@netronome.com @netronome www.netronome.com

Machine-to-Machine (M2M) communication, or the ability to connect and manage remote devices over the air, offers enormous potential. With the ability to centrally control remote industrial equipment, track vehicle fleets, manage electric vehicle charging stations, expand the capabilities of consumer devices, and much more, M2M has profound implications for virtually every industry.

Given the novelty of M2M technology, however, developing connected devices has traditionally been an expensive and time-consuming process, largely due to the fact that system designers had to build the entire M2M architecture from scratch. Today, designers have a powerful new option in their M2M toolkit: Embedded Application Frameworks (EAFs). By deploying connected services on mature, prepackaged Real-Time Operating Systems (RTOSs) and libraries embedded directly into the communications module, M2M designers can substantially reduce the time and costs involved in developing new M2M hardware and focus their efforts on creating innovative connected applications.

Challenges of developing M2M systems

At its core, M2M technology involves augmenting a device or piece of equipment with intelligent services and connecting that device to a back-end infrastructure that can monitor or control it. To accomplish this, an M2M device employs two basic elements: a mechanism to communicate with the back-end infrastructure (a wireless modem or module) and software to run the services.

Mature wireless communication modules have been available for many years, and designers of connected devices have often used off-the-shelf components to provide connectivity. Most of them relied on a traditional multichip architecture. In practice, this required assembling the hardware and software – usually based on a full-blown OS along with its associated software libraries, running on a stand-alone microprocessor supported by external memory – before designers could even begin addressing the services running on top of the device.

Early developers of connected devices had few choices available because there simply wasn’t a mature market of prepackaged software available for supporting M2M connectivity. But this reliance on bespoke device architectures introduced a number of inefficiencies that today’s developers can no longer afford.

Developing systems in this manner takes a long time. Assembling and integrating the entire architecture from scratch typically requires a minimum of one year in development before the system can be brought to market. While this might have been an acceptable timeframe in the early days of M2M, system providers today cannot afford to wait that long. They need to stake their position in the marketplace as quickly as possible.

Building the entire architecture from scratch is also inherently expensive. Apart from the operational costs associated with integrating and testing all components of the architecture in-house, relying on this model also typically involves using a full-blown RTOS and having to equip the device with full-scale processing power to run it. Some complex M2M applications require this much horsepower, but for the vast majority (which often simply monitor a device and send out data to a back-end server), a full-blown RTOS is overkill. Why invest in a full-scale OS and microprocessor, when what the device is actually doing requires just a fraction of that computational capability?

The biggest drawback of this approach, however, is that it requires connected device developers – often at a start-up company – to devote significant time and resources to things that have nothing to do with their core areas of expertise. For example, if a developer is building an M2M system for health care, the value of that system lies in the intelligence created for a specific health care application. A developer of industrial systems possesses expertise in developing services that effectively monitor and control that equipment using the most appropriate protocols. Whatever the industry or expertise, a connected device developer’s core value proposition is most likely not assembling multichip computing architectures.

Embedding intelligence in the communication module

EAFs address these issues by providing a means to embed M2M services directly into the communication module, alongside blocks of prepackaged software, connectivity capabilities, and processing resources (see Figure 1). In this way, an EAF makes it easier, faster, and less expensive to deploy an M2M system. It allows developers to use mature, proven, widely deployed technology instead of having to reinvent it. EAFs improve:

·    Time to market: By using prepackaged components and embedding software code directly in the communication module, M2M system providers can substantially reduce development timelines. Instead of taking a year to develop all aspects of the system, many M2M applications can be developed and brought the market in less than six months.

·    Development costs: Deploying software in an EAF on the communication module eliminates the need to buy and assemble a separate RTOS, microprocessor, and external memory for a device. Because an EAF provides a lightweight OS specifically optimized to run common M2M services, it can share processing resources and memory with the communication module. This also reduces operational expenses by eliminating the need to staff engineers with expertise in OSs and communications, and instead focus engineering resources on the application and its unique services.

·    Efficiency: Relying on a multichip architecture with a separate communication module and microprocessor limits the RTOS to a relatively simple command interface with the modem. When the application is embedded on an EAF in the communications module, however, it can directly access all the different layers of the communications stack. That means the developer has more control over how the application monitors and accesses the communications stack at different levels using different APIs. It also delivers capabilities beyond those available to a stand-alone RTOS.

(click graphic to zoom by 1.5x)

Most importantly, EAFs address the “core-versus-context” question, allowing connected device manufacturers to focus on the unique value they bring to the system.

Key elements of an EAF

So what should connected device manufacturers look for when considering the EAF model? Any EAF should include the following core components:

·    Lightweight OS optimized for M2M: While a few M2M applications require a more powerful RTOS, most do not. The keys for the EAF OS are a small footprint and ruggedization for M2M deployments. The OS should be natively designed to provide APIs that control voice call, data call, SMS, and TCP/IP connectivity. It should be optimized to take full advantage of its direct access to the communications stack. To provide full support for a connected application, the OS should also provide a core feature set that includes:

-  Real time, including guaranteed response time to external or internal interruptions, regardless of its state.

-  Flexibility to prioritize tasks.

-  Multitask capabilities to define and synchronize as many tasks as services require.

-  Flexibility in processing speeds and power options to optimize battery life.

-  Memory, firmware, and software protection features.

-  Ability to use APIs to access the cellular modem’s audio and data path.

·    Software libraries: To simplify the development process and speed time to market, the EAF should include a variety of software libraries and APIs that provide a variety of functions the device or services might need. This includes services such as location/GPS connectivity, comprehensive Internet connectivity protocols, and wireless and Internet security services. The EAF should also support third-party libraries that take advantage of software developed for the specific needs of the target market. Ideally, the EAF should be backed not just by the communication module vendor, but also by a community of partners and developers working to expand its capabilities.

·    Development tools: The EAF should also include a package of development tools that make it easy to code, debug, and monitor M2M applications, and these tools should be open source and free to use. Ultimately, the EAF should provide everything needed to develop and embed the M2M application into the module.

·    Cloud connectivity: Finally, the EAF should provide tools to streamline cloud-based management of connected devices, including a fully realized system to handle device monitoring and software/firmware upgrades over the air. The system should allow developers to monitor the health of the devices and identify potential problems. It should also include proven tools to remotely upgrade the OS stack, as well as the M2M application itself using a patch mechanism.

Taking advantage of prepackaged M2M components

The EAF discussion raises an age-old question for businesses: Should I make it or buy it? For most companies and most M2M systems, buying makes a good deal of sense. After all, if you were starting a car company, would you make your own tires and windshield wipers? Your own stereos and navigation systems? Clearly, there are some elements of the product developers will want to build themselves, as that’s where they can add the unique value that differentiates their system. But for most of the M2M architecture, the market now offers mature prepackaged components that are both proven and cost-effective.

Some companies have concerns about ceding control over the system when using prepackaged M2M components. However, the reality is that lack of control should not be an issue with any modern EAF. As long as the EAF supports open standards, developers should be able to write code in a common programming language such as C/C++, which means they retain the ability to port that code to any other platform used in the future.

Of course, there will always be exceptions to this rule of thumb. For some companies and projects, it makes sense to build everything in-house, as the product’s value lies in reinventing the entire system. There are also M2M systems that are simply not suitable to run on an EAF – more complex, heavier applications that require the horsepower of a more powerful processor and a full OS.

Even this distinction might not be relevant for long. Today’s EAFs rely on previous-generation processors that have been on the market for several years. As EAFs continue to evolve and take advantage of higher-powered processors and multicore architectures, even companies developing very complex M2M applications will likely be able to embed them into the communications module EAF and benefit from the same advantages.

As EAFs and the M2M market continue to evolve, there will be fewer and fewer reasons for connected device manufacturers to invest in building basic M2M capabilities, much less entire device architectures. Ultimately, this will make system design much easier for M2M developers. More importantly, as designers focus more on delivering unique value instead of on M2M hardware, we can expect to see M2M innovations that at present can only be imagined.

Pierre Teyssier is senior VP of engineering for the M2M Embedded Solutions Business Unit at Sierra Wireless.

Sierra Wireless
PTeyssier@sierrawireless.com
Twitter: @SierraWireless
www.sierrawireless.com

Traffic in the enterprise and carrier network has exploded in recent years, driven by consumer broadband, corporate traffic, and newer IP-based services such as mobile connectivity, remote cloud services, IP video, and IPTV.

In addition, the advent of virtualization and the need for higher-performance (up to 100G) secure communication have put tremendous pressure on communication system designs, including the I/O subsystem. These demands, coupled with the success multicore x86 CPUs have had in embedded applications and the data center, have created the need for a coprocessor that can handle packet processing at tens of millions of stateful flows with a glueless, high-performance, virtualized interface to the x86 CPU subsystem.

The pressure of packet processing

Today, service providers offering cloud-based services and enterprise data centers are enabling access to valuable resources anytime, anywhere over wireline and wireless networks. The resulting increase in traffic is putting aggregation switches/routers and intermediate network nodes under constant pressure to meet ever-higher bandwidth demands. These processing elements do not simply switch or route traffic; they must also perform functions such as building a firewall with Deep Packet Inspection (DPI) capability and offering virtualization support for multi-tenant cloud environments.

The underlying Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Real-time Transport Protocol (RTP) traffic comprises many packets belonging to a network connection. An intermediate node such as a switch, router, or gateway at the network edge must process millions of network connections simultaneously.

Trying to process each packet in the connection individually inhibits the network element from being able to keep up with ever-increasing line rates. This is further complicated by the need to perform DPI for at least a portion of the traffic. Moreover, an intermediate node located deep in the network must process hundreds of millions of packets.

Each packet has no correlation with any other packet; that is, they are not related in space or time. Such asynchronous traffic is better served by grouping packets into flows. A flow is a collection of packets belonging to the same network session, usually between a source-destination pair. Incoming packets must be classified into flows. The processor then deals with all packets belonging to the same flow in the same way based on rules in a flow state table.

Stateful flow processing

All network elements require states for millions of flows, especially when it comes to implementing security processing such as firewalls, intrusion prevention or detection systems, and application-level load balancers. The resulting platform architecture must support flow state management by monitoring packets within a flow, updating a TCP connection, creating and timing out UDP connections, and tracking Virtual Private Network (VPN) connections. State handling is also required to support TCP proxy and TCP splicing.

System software should thus maintain flow state tables supporting millions of flows. Hardware must support software by performing a complex hash and lookup in a flow hash table. Software is responsible for analyzing the flow hash result and managing new flows, updating the hash table and maintaining the flows’ state.

System performance requirements at 100G

To meet the stringent system requirements at 100G, both processing and memory architectures must meet the time budget offered by one packet time at the worst case of 64-byte packets, which is as low as 5 ns.

Processing instruction and memory budgets

Given that most networks continue to use Ethernet frames or packets as the underlying transport, it is important to understand the composition of these frames and how they affect network performance.

The Ethernet frame

A typical Ethernet frame starts with an 8-byte preamble, followed by 12 bytes of addressing information for destination and source addresses, a 2-byte type/length field indicating the type of date used, and the length of the payload. The payload data can be as low as 46 bytes and as high as 1,500 bytes. A 32-bit (4-byte) cyclic redundancy check is computed and appended at the end of the frame (Figure 1).

(click graphic to zoom by 1.9x)

Performance calculations at 100 GbE

System throughput calculation is usually expressed in a packets-per-second (pps) figure. The maximum number is calculated when all Ethernet frames are 64 bytes in length, or the minimum size frame. For 10 GbE, this number is 14,881 million pps, or commonly known as 15 Mpps. For 100 GbE, this number becomes roughly 150 Mpps.

Smaller packets present a challenge in meeting the short time budget, while large packets present a challenge in meeting the highest line rate. The per-packet time budget required to process a 64-byte packet is as little as 6 ns. With a processor running at 1 GHz, the instruction cycle time is 1 ns. Hence, a 64-byte packet translates into a 6-cycle budget at 150 Mpps. One way to get around this constraint is to use parallel processing with multiple cores and threads. For example, a 100 cores/threads processor will increase this time budget to 600 cycles – a far more manageable window.

Memory considerations at 100G

The use of specialized memories is not recommended in networking devices. At present, DDR3 memories are the preferred external memories. DDR memories operate well in longer bursts; however, transaction rates for clocks higher than 1,666 MHz reach maximum rate for 64-bit wide interfaces. Exchanging a 64-bit channel for two 32-bit memory channels can deliver higher transaction rates at clock frequencies of 2,133 MHz and higher.

Current approaches to fulfill 100G requirements

Multicore processors

In the early 2000s, many new and established chip vendors started offering multicore CPU products based on standard general-purpose processors, creating Symmetric Multi-Processing (SMP) Linux structures. In leveraging the relatively simple programming model for SMP Operating Systems (OSs), networking vendors were able to introduce products to market in less time. However, this approach was limited to sub-10G levels of performance.

Performance in these processors is limited primarily because traditional general-purpose CPUs have relied on caches to work around memory latency issues. Cache misses force the CPU cores to starve for memory accesses, where main memory latency is way too slow compared to cache memory. This so-called “memory wall effect” implies that the SMP multicore model for processors does not scale to the hundreds of processor cores required to flexibly address 100 Gbps solutions. Attempts to minimize cache misses through branch prediction and speculative execution techniques fall short of solving the relatively low-cache hit-rate problem.

In an attempt to circumvent the performance bottleneck, vendors began embedding hardware accelerators into multicore processors to handle common performance-intensive functions such as security and DPI (see Figure 2). The resulting single-chip heterogeneous multicore processor has given way to proprietary architectures that are not OS friendly, and has defeated the original intent in having a simple, easy-to-program multicore processor.

(click graphic to zoom by 1.8x)

Network processors

Network processors are a category of processors focused on optimizing L2-L4 packet performance. In general, they contain smaller cores that scale reasonably well and can deliver 100 Gbps of performance. Memory performance is addressed through pipeline architectures, and in some cases, Very Long Instruction Word (VLIW) architectures.

Flexibility and intelligent processing are hampered in network processors due to complex programming and fixed internal structures focused at packet forwarding. Furthermore, performance in pipelined network processors suffers when traffic consists of several tunnels and/or when deeper tunnels are required.

Ethernet switches

This category of chips typically includes small pipelines with internal lookup engines and does not support external memory. Usage was common in enterprise Ethernet wiring closet switches. As the usage models grew in complexity with top-of-rack switches, the flexibility requirements also became more pronounced. Larger lookup tables and greater performance levels are now required from Ethernet switches, as well as several deep tunnels needed to support the many layers of virtualization in the data center.

Although some Ethernet switching chips have access to external ternary content-addressable memory for fast table lookups, a typical Ethernet switch can’t access external DDR memory, making it difficult to cater to networking applications that require support for millions of flows.

Ethernet controllers

This category of products is used in server and client environments to connect multiple Ethernet interfaces to the host x86 CPU through a PCI Express interface. These devices can’t be programmed to perform complex networking tasks such as switching or in-line security. They have no access to external memory and hence can’t support millions of flows.

Now that the challenges of processing network traffic at 100G have been identified, it is important to discuss what is needed to address these challenges. Part 2 of this series, which will be featured in the February issue of Embedded Computing Design, will highlight the need for a coprocessor that can meet the challenges that arise with 100G network traffic. Additionally, the second article will discuss how the new coprocessor manages functions such as intelligent L2/L3 switching, flow classification, in-line security processing, virtualization, and load balancing for x86 CPU cores and virtual machines.

Nabil G. Damouny is senior director of strategic marketing at Netronome.

Netronome
408-496-0022
info@netronome.com
Twitter: @netronome
www.netronome.com

What is the focus of the division you work for at Freescale?

BECK: I’m with the Network Products Division. We have responsibility for Power Architecture technology, where we build processors primarily for the networking and telecom marketplace as well as the general industrial marketplace. Because our products offer specific blends of performance and energy efficiency, Freescale’s processors are well established within the aerospace and defense markets as well. Whether it is in flight controllers, graphic displays, or radar imaging, we’re involved in it all.

I understand that Freescale recently introduced its new QorIQ AMP Series. Aren’t they the processors with AltiVec technology reinstated?

BECK: Yes. We created a family of processors called QorIQ communications platforms, beginning with the P Series family, based on 45 nanometer process technologies. We have products that range from less than 5 W to a maximum of 30 W. Just recently, we announced the new QorIQ AMP Series products, based on 28 nanometer technology. This new family of QorIQ AMP processors is a continuation of multicore processing platforms introduced as QorIQ communications platforms. Clearly, the use of many-core microprocessors is how the industry can deliver more performance within embedded power envelopes. Freescale’s trust architecture is also designed into the AMP processors, providing the ability to securely boot up the device, provide domain separation, and detect external threats to one’s system. In addition, the AMP series has reintroduced AltiVec technology, which is available on our new e600 core. AltiVec is used in aerospace and defense applications as a signal processor within a control/communications processor. It is well adapted in radar and graphics display imaging – basically, in any signal processing application.

Okay, let’s talk AltiVec technology for a moment, which has been a hot topic in a lot of the discussions I have been sitting in on. AltiVec went away for a while, and now it’s back in this AMP Series processor. What was the impetus?

BECK: Right, so everybody remembers that AltiVec technology was first introduced in the MPC7400 series processors. That core later became known as the e600 core when it became part of a System on Chip (SoC) platform. The first introduction of the QorIQ family used the e500 core, which didn’t include AltiVec technology. There is no doubt that the aerospace and defense market has had strong adoption of AltiVec technology for many years, and frankly it was missed by this market. But over the past three to four years, we have seen an increased demand and requirement for SIMD-like performance in other markets outside aerospace and defense. Like a lot of companies, we are market- and customer-driven and we began to see a need for AltiVec in the telecom, networking, video surveillance, and medical imaging spaces, to name just a few. As those appeared, we knew it was appropriate to bring this technology back to the QorIQ family of processors. That reintroduction has occurred in the recently announced QorIQ AMP T4240 product.

Looking forward, do you think all of your processors will have AltiVec on them?

BECK: As far as I can see for the foreseeable future, there is no reason for us to not include the technology in our QorIQ products. In fact, I think we expect constant improvements in the future.

Are there any specific road maps for improving AltiVec?

BECK: None that I can speak to at this time; however, many choices will be available to customers. If you remember, we first introduced AltiVec in the MPC7400. Then in about a year and a half, there was an MPC7410. And in another year and a half, there was an MPC7445, and on and on until we came to MPC7448. So about every year and a half we would have a new device with AltiVec technology incorporated, delivering ever-increasing performance. With the QorIQ AMP series, one can expect to see a number of devices across the T1 to T5 series over the next 12 to 24 months. This will provide customers more choices across a broad range of power, performance, and cost and greater alternatives for all kinds of applications.

Earlier you mentioned the AMP Series facing challenges of more performance in power envelopes and security issues. Let’s address those.

BECK: Whether it is the soldier on the ground or in the plane in the sky, the amount of voice, data, and video processing required is rising astronomically. In addition, more systems are becoming unmanned, whether they are aerial or ground vehicles. These UAV systems are requiring ever-increasing sensing capability for growing data analysis. The result is a need for more autonomous real-time decision capability at the point of data collection. This translates directly to the need of more processing capability within very demanding power (heat) envelopes.

Classically, the way we have provided better performance is by decreasing the size of transistors, which means you’re able to switch transistors quicker and therefore deliver more frequency. We complement this with increasing performance through system functionality. We have moved from PCI at 66 MHz to PCI Express interconnect at 1 and 2.5 GHz and PCI Gen2 at 5 GHz and are moving from DDR1 to DDR3 memory controllers. All of those help system performance. I do not know about you, but it seems this transition has occurred in such a short period of time. At Freescale we design SoC microprocessors that balance performance across cores, memory, and I/O subsystems in a single device at particular embedded power envelopes (Figure 1). This platform strategy provides customers alternatives to get the maximum performance for their applications.

(click graphic to zoom by 1.9x)

The other way you increase system performance is by increasing the frequency and the number of cores. Of course, the challenge associated with this is increased power and heat. We designed the QorIQ platforms devices within designated power envelopes. For example, the P1 family is a 5 W envelope, P2 is 10 W, P3 is a 15 W envelope, and P4 and P5 devices are 30 W envelopes. All these P series processors are 45 nanometer products.

Those are the original QorIQ processors, but can you give a technical example of this improved power envelope for the AMP Series?

BECK: As a design philosophy we balance SoC devices that utilize cores, hardware accelerators, memory, and I/O subsystems. The T4240 uses the new e6500 Power Architecture core. This 64-bit core is dual threaded, with L1 and L2 cache and also contains the 128-bit AltiVec unit. These cores connect inside an SoC platform via a coherency fabric that allows us to create point-to-point connections between I/O, accelerators, and memory subsystems resulting in high-performance processors within embedded power envelopes. We take advantage of hardware accelerators to offload work on cores, which allows us to accomplish work with fewer transistors, translating to lower power. Examples of those hardware accelerators are security engines and pattern-matching engines. As we move from the P4080 device to the QorIQ AMP T4240, we are delivering more performance per watt than previously realized. Benchmarks are demonstrating a 4x performance – pretty exciting stuff.

Where do the higher speeds come from?

BECK: The industry always speaks about improved speed, but it’s really about improved performance. But first, let’s address the speed. As has been done for more than 20 years, we get improved frequency with improved process geometries. We had improvement in frequency as we went from 1.5 GHz up to 2 GHz as we moved from 45 to 28 nanometer technologies while maintaining the same pipeline architecture. Additional performance improvement resulted from increasing the number of cores from 8 to 24 virtual cores. And we provide significant floating point performance with use of the AltiVec unit where the T4240 is capable of generating 192 GFLOPS.

Let’s move on to the security you mentioned earlier – where does that fit into the AMP Series of QorIQ?

BECK: I use an acronym that is applicable to the A&D market. The addition of a capital “A” to “SWaP,” which makes it “SWAaP.” It’s about adding “Assured computing” to the long-held requirements related to Size, Weight, and Power. And it has become another critical component to every system. Every A&D RFQ or RFI has a secure or assured component. The delivery of that component cannot jeopardize the performance requirement of the mission. All future systems must be capable of withstanding security attacks. Those attacks come in the form of:

1) Theft of functionality: When someone is able to take over that system and cause it to act in ways that it wasn’t intended to behave.

2) Theft of uniqueness: OEMs spend millions of dollars and years of development of systems. To see those investments efforts be reverse engineered can be disastrous.

3) Theft of Data: Loss of one’s IP or data that is stored on the device to unauthorized parties.

How are these challenges mitigated in the AMP Series?

BECK: We created our trust architecture capability in the P series product and have extended it in the AMP series. The primary features are secure boot, domain separation, tamper detection, and secure debug. In terms of a secure boot, you want to know that your bootable image is trusted and has not been tampered with. So, we provide the user the ability to put a secure key fused into the device. The boot process begins within the walls of your factory. With a private key, you create a 256-bit hash signature of your known trusted image. During the secure boot process, you create a signature with the key you have stored on the processor and validate that signature to the image while booting the processor. If validated, the processor is brought up in a trusted state.

Once in a trusted state, you can then begin the process of creating domain separation of cores, memory, and I/O devices. In other words, you want to distinctly define a particular relationship between cores, memory, and I/O devices in a defined partition. Domains are now able to operate independently without fear of corruption. We have intentionally designed the capability into the processor to ensure that domains are separated in the way the user wants them. This allows customers to provide separation between public, confidential, and secret regions.

Trust architecture also has the ability to react to physical threats to the system. The types of attacks are defined by the OEM. Some may be as simple as opening up a cabinet door. When this happens, a signal is sent to the processor and it immediately begins to zeroize the memory regions of the device. The device is now unusable and can only be made whole again through the secure boot process.

What about network attacks?

BECK: The way we prevent network attacks is decided by the user’s security policies during the secure boot process though the partitioning of the cores, memory, and I/O. Each partition is identified with a unique logical address. Network attacks come in the form of when someone will attempt to insert harmful instruction code in a particular memory region to gain control of a device’s functionality. Network packets must begin authorization through a secure tunnel to the partition. Unauthorized requests are prevented from accessing a partition. A security monitor notes these attacks and notifies the OEM.

In addition, one must be able to have access to the system once it is deployed in the field. One is able to securely debug a processor, provide software upgrades, and change security policies while the system is in the field. This is done through a debug port that is entered via a question/answer handshake that the OEM has created and stored in the processor.

In the past, these types of security features [secure boot, threat detection, secure debug, and domain separation] have been external to the processor by FPGAs, TPMs [Trusted Platform Modules], or even custom devices. We have put these features inside the QorIQ processors. It has the benefit of increasing security by fewer devices and exposed buses. In addition, it decreases the overall system cost.

Is the trust architecture user-friendly, or is it complex to navigate?

BECK: We believe we have made the implementation of the trust features user friendly. But implementation of the trust architecture is a big commitment for the user. If you ever lose your private key, it makes those systems unusable. It is a user-defined option. If you choose not to put it in trust mode, then you can use it as any processor is used today.

One last question: What is needed now in your arena, but not available now?

BECK: It is interesting to me that there was a real fear about how quickly multicore processing would get adopted. While there is considerable need for improved software tools, the adoption has been extraordinary. The ever-increasing demand for performance within stringent power envelopes has resulted in a broad spectrum of applications. It is clearly the path we are all taking. We have just begun to deal with the challenge of certifying multicore systems in commercial aviation. While challenging, it will happen.

In addition, the fundamental challenges of system design remain: latency, bandwidth, and power. I know here at Freescale, we are constantly working to solve these real-world problems. Frankly, it’s lots of fun.

Glenn Beck is Market Segment Manager for the Network Products Division, Freescale Aerospace and Defense/Single Board Computing Market. He is a 32-year semiconductor veteran with Motorola/Freescale. His experience has been in design and product engineering, and he presently leads Freescale’s marketing efforts in A&D and single board computing with Power Architecture microprocessors. He holds a BS in Electrical Engineering from Texas A&M University and an MBA from Texas State University. He leads the Multicore for Avionics (MCFA) working group of avionics suppliers in an effort to certify platforms with multicore processors. He can be contacted at glenn.beck@freescale.com.

Freescale Semiconductor 512-996-5043 www.freescale.com

Transportation means movement. Movement means variability. Variability means design challenges. When building a small form factor embedded computer for transportation applications, there are inherent design challenges regardless of application-specific requirements. In fact, it can be argued that small form factor design trends are paradoxical. As form factor size decreases, functionality requirements increase. And at the same time that processing power requirements heighten, lower power consumption and lower thermal output are expected. Now add to that the requirement for ruggedness to accommodate for the shock, vibration, humidity, and temperature extremes and variance inherent in mobile and outdoor applications; developers are often left wondering where they can safely compromise while still meeting overall application design specifications. Knowing the application’s needs and what already exists on the market that fits those requirements makes transportation design less daunting.

Choosing formats

PC/104 and Embedded Board eXpandable (EBX) are good format options for designs that can handle slightly larger Single Board Computer (SBC) form factors. Intended for data acquisition in rugged environments, the PC/104 embedded computing format has no backplane, instead allowing modules to stack together like building blocks – more rugged than typical bus connections in PCs. And with just 46 square inches of surface area (8" x 5.75"), EBX balances size and functionality with a bolt-down SBC format, supporting rugged embedded designs with higher-performance Central Processing Units (CPUs) – such as those using multicore technology for networking, Digital Signal Processing (DSP), and graphics-heavy applications – and generous onboard Input/Output (I/O) functions to support everything from large data exchange to video.

However, the most extendable and customizable application design method accommodates a broad range of custom and off-the-shelf needs by using modularity. Computer-On-Modules (COMs) are complete embedded computers built on a single circuit board for use in small or specialized applications requiring low power consumption or small physical size. Though they are compact (ETX/XTX at 114 mm x 95 mm and COM Express at 125 mm x 95 mm) and highly integrated, COMs can accommodate complex CPUs.

With the COM approach, all generic PC functions are readily available in an off-the-shelf core module. A custom-designed carrier board complements the COM with additional functionality that is required for specific applications. The carrier board provides all the interface connectors for peripherals, such as storage, Ethernet, keyboard/mouse, and display. This modularity allows the designer to upgrade the COM on the carrier board without changing any other board design features, and also allows more customization of peripherals as dictated by a specific application.

COMs allow system developers to focus on their core competencies and the unique functions of their systems. The COM Express form factor offers flexibility in the development and advancement of ultra-rugged embedded applications for a plethora of industries, including transportation. By using the modular processing block, the designer creates a price and value advantage; he/she isn’t locked into a single vendor for board creation and can customize based on pricing and performance requirements. Because it is easily swapped from a carrier board and comes in one of the smallest form factors, COM Express is ideal for long-life embedded applications with a critical development cycle, as well as more progressive applications that require frequent processor upgrades without affecting other application design elements.

Designing for harsh environments in transportation

Transportation solutions are most often housed outdoors or in moving vehicles, where exposure to a variety of climates dictates the need to operate in extended temperatures and to power up in any extreme. The easiest initial step is to select a rugged board or system that is designed for harsh environments from the ground up. To support the extremes of shock, vibration, humidity, and temperature, care is given to component selection, circuit design, Printed Circuit Board (PCB) layout and materials, thermal solutions, enclosure design, and manufacturing process. Robust test methods, including Highly Accelerated Life Testing (HALT), ensure optimal product design phases in order to meet a product’s stringent requirements, such as -40 °C to +85 °C operating temperature range, MIL-STD, shock and vibration, and long-term reliability.

Onboard train systems also deal with high concentrations of sulfur and humidity when going through tunnels. Designers can look for boards with conformal coating to reduce degradation from such exposure. Conformal coating is used in small form factor manufacturing rather than potting, which is a similar process that uses a heavier material and is harder to inspect, test, and repair. Though considered the highest level of environmental protection, potting encapsulates the entire PCB, which adds weight and expands the dimensions of a unit. Even an extra millimeter can be critical in small form factor design, which is why conformal coating – with a single-part material that conforms to the board – is a better option. A variety of conformal coating materials (such as acrylic, polyurethane, epoxy, and silicone) and application methods (such as brushing, spraying, and dipping) are currently used to protect against moisture, dust, chemicals, and temperature extremes that can potentially damage electronics. The correct coating or application method varies depending on established standard operating conditions for an application. With transportation applications, different coatings may be selected based on a primary need for moisture resistance versus abrasion resistance versus temperature stability.

Maintaining performance while mobile

Transportation applications typically need as much functionality as possible in the smallest form factor, meaning the controller may be burdened with extreme loads of information and intricate tasks. Rugged computing solutions also demand more memory space than ever before for both data storage and application performance. Options for storage include rotating Hard Disk Drives (HDDs) for economy or Solid-State Drives (SSDs), which are truly rugged, but also come at a higher price point (cents per GB for HDDs versus dollars per GB for SSDs). SSD microchips that retain data in non-volatile memory chips and an absence of moving parts make them less susceptible to physical shock, altitude, and vibration issues. SSDs have faster access time and lower latency than do HDDs, but SSDs cannot provide the capacity of an HDD; because of the higher cost per GB, SSDs are typically no larger than 120 GB, while HDDs average 500 GB to 1 TB. Higher performing HDDs also require heavier materials than either a standard HDD or the flash memory and circuit board materials of SSDs.

With in-vehicle surveillance applications, vibration control is critical for capturing quality video. Some rugged boards offer a thicker PCB fabrication to add rigidity so the board can withstand higher levels of vibration strain. The thicker PCB offers stability to the overall surface area, protecting electronic components from damage due to vibration. The thicker PCB also offers the ability to use more copper between layers for thermal considerations. Heat is a common unwanted by-product of processing power. In addition to cooling fans and large heat sinks, which may not always be usable for compact, mobile transportation designs, PCBs with adequate amounts of integrated copper facilitate heat conduction away from temperature-sensitive electronic components to prevent performance degradation.

Another challenge with designing in small form factors is that current power supply technology can put limits on size reduction. Rugged designs, specifically, require a robust onboard power controller to support a wide operating temperature range – on average of -20 °C to +70 °C. An onboard power controller is also critical for mobile transportation applications to support multiple data usage requirements, such as collecting video, remotely checking the health of onboard system devices, and sending command controls.

Balancing connections

For an in-vehicle or outdoor video/audio capture application design, the board itself needs high-performance graphics and host interface support for multiple peripherals. In addition to the integration of a video camera, features such as remote monitoring and wireless video download call for some form of connectivity. Both satellite and cellular connectivity require either an antenna or a device with an antenna to connect directly to the system. Ample Ethernet and serial interface ports – with at least one port supporting RS-232/422/485 for Transmit and Receive – are critical to accommodate function-specific peripherals.

Case study: Onboard locomotive video system

A leading global supplier of technology solutions for railroads wanted to develop an onboard locomotive video/audio capture system to aid in accident investigations and provide safety training to crews. In addition to video and audio recording, requirements for the system included remote monitoring and control, real-time health monitoring, and wireless video download. The system also needed to incorporate solid-state media in a sealed, tamper-resistant housing.

Ampro by ADLINK is a line of extreme rugged embedded computers and systems that provides designers of rugged applications with a head start to development. For the onboard locomotive video/audio capture system, designers created a rugged solution around the Intel embedded architecture and EBX SBC form factor. The design accommodated both the functionality and rugged requirements for the train system with dual Ethernet, CRT and flat panel video, multiple serial and USB ports, SATA and IDE interfaces, CompactFlash socket, PCIe Mini Card socket, high-definition audio, and General Purpose Input/Output (GPIO) support.

A critical byproduct of on-site video/audio capture is reduced litigation and settlement costs due to accurate incident reporting. System reliability is critical to users in terms of Return On Investment (ROI), so designers should build using products that provide documented uptime in their specifications and meet the industry standard of EN50155. Because of the cost and complex nature of embedded computing solutions for transportation, qualification can take a very long time, requiring designers to look for products with a long lifecycle. One way to ensure system longevity is to build with products that follow the roadmap of an established architecture.

Optimal designs for transportation

Though designing for rugged small form factor applications presents challenges, there are form factors and manufacturing techniques that accommodate most requirements for either general or application-specific design. Formats such as PC/104, EBX, and COMs have been created specifically to address rugged embedded system needs, while also handling complex CPU technology for applications that require heavy processing power. Modularity also helps designers create customizations while taking into account cost and value requirements. The bottom line for any rugged design, whether for transportation, military, energy, or infrastructure, is to understand all application requirements and how/where existing formats and products can address those needs. By selecting the right core and rugged framework, focusing on and prioritizing any added customization becomes a much simpler mission.

Jeff Munch is CTO of ADLINK Technology and heads all R&D operations in North America and Asia. Jeff has more than 20 years of experience in hardware design, software development, and engineering resource management. Before joining the company, he spent five years at Motorola Computer Group as Director of Engineering. Jeff is Chair of the COM Express R2.1 Subcommittee, Chair of the AdvancedTCA Subcommittee, and Chairman of the PICMG COM Express Plug-and-Play Subcommittee.

ADLINK Technology Jeff.munch@adlinktech.com www.adlinktech.com

For more than 60 years – from the early days of mainframe computers through the PC revolution of the 1980s and into today’s explosion of smart mobile devices – processor technology has always evolved to meet users’ expectations, at times driving unforeseen innovations in the computing industry. Given the diversity of new mobile devices coming to market on a daily basis, processor innovations continue to be a powerful force for change.

With the advent of mainstream mobile computing, processor architectures have shifted from the traditional desktop model driven by performance, regardless of the power required. Devices that require all-day or even multiday battery life will require a more compact energy envelope while pushing processor performance to new levels.

In the beginning, performance was king

The first consumer computers powered by microprocessors were simple, power-hungry, stationary devices that were tethered to their source of electricity (the common wall socket). This meant that microprocessors could be designed solely with performance in mind, which soon became the “holy grail” for developers.

Early PCs comprised a single-threaded CPU running a single application. Soon, these early 8-bit microprocessors grew to 16-bit and, eventually, 32-bit processing by the mid-1980s. Then the market started to see PCs capable of running multiple applications simultaneously. With performance rising as the number of transistors doubled in cadence with Moore’s Law, each new processor design offered the ability to develop new features and functionality, whether playing a DVD or editing the family album, which, in turn, whetted consumer appetites for more powerful devices.

Eventually, consumer demand for different form factors and the expectation of performance improvements pushed the demands on processors beyond what was capable within a single core. At the same time, demand for mobile devices started to explode and, as it grew, so did the call for more energy-efficient processing.

When ARM was launched in 1990, a main objective of its founders was to create an energy-efficient processor architecture for handheld devices. Employing the RISC CPU architecture, ARM’s approach simplified instructions, streamlined task execution, and reduced the power required per instruction.

A few characteristics were critical to the development of a more energy-efficient microprocessor, the most important of which was an intense focus on limiting power consumption to the lowest possible levels. Keeping the power envelope to the smallest possible footprint not only increased battery life, but also limited the weight of the battery required to power the device, which reduced the bill of materials and kept overall costs down.

Multicore: Benefits beyond mobile

Today, the benefits of this high-performance, energy-efficient processing architecture are bearing fruit in devices such as digital televisions and set-top boxes, office equipment such as printers and copiers, and mobile devices such as tablets, portable gaming units, and smartphones.

Since the mid-2000s it has been accepted that building bigger and bigger CPUs to realize single-thread performance gains not only becomes increasingly difficult, but also runs counter to the energy-efficiency limitations of mobile devices. This is because exponentially more energy is required for every few percentage points gained in performance.

Multicore solutions can deliver higher performance at comparable frequencies to single-core designs while offering dramatic savings in terms of cost and power efficiency. Furthermore, multicore solutions can leverage cores with high transistor counts and optimize systems by powering them up only when needed. In essence, this can be thought of as intelligent load balancing. Not only does a system need to consider which processor is best suited to execute a specific task, but it also must consider the performance required of that task and assign it to the most power-efficient processor available.

Using cores as needed while keeping others idle helps keep energy consumption as low as possible, with limited impact on performance. As tasks are distributed across multiple processor cores, an individual processor might not run at full capacity, allowing the voltage and frequency of a multicore processor to be lowered. This results in significant power savings related to the system’s aggregate performance.

To illustrate this idea with a common use case, consider today’s smartphones, which must be powerful enough to render a complex Web page and play gaming applications, often in parallel with basic e-mail synchronization and phone management functionality. With the ability to power up cores only when needed, multicore smartphones can deliver increased battery life compared to their single-core, full-throttle predecessors. Market demand for scalable performance has resulted in most current smartphones containing multicore CPUs, as well as GPUs with multiple cores found in many of today’s leading mobile video and gaming devices (see Figure 1).

(click graphic to zoom by 1.9x)

A “many-core” approach to multicore processing requires the performance loads to be shared across many smaller processors, such as a Cortex-A5, rather than with multiple single-thread workloads across a single-core processor. Designers are increasingly deploying clusters of processors designed to work together, sharing data and tasks among caches or multiple instances of the same processor (see Figure 2).

(click graphic to zoom by 1.9x)

Many core becomes even more interesting as smaller processors work together to deliver a combined performance level with lower power consumption than a larger processor multitasking the same workload. As previously mentioned, the costs associated with increased performance on a single thread are exponential; however, with multicore processing the cost becomes more linear in scale. Designers are using many cores to significantly reduce aggregate system costs.

As hardware designers begin to implement these many-core systems, software developers will need to produce code capable of using a many-core processing solution. Until then, devices must have the ability to execute high-performance tasks. One example of a system that contains both high single-thread performance multicore and the greater power efficiency of many cores is the current deployment of CPUs and GPUs, where the many-core GPU can deliver graphical computation using less power than the multicore CPU. Since the GPU remains coherent with the CPU and shares its caches, external memory bandwidth and performance demands on the CPU can be reduced. Languages such as OpenCL and CUDA are working to enable these issues for more generic applications.

Optimizing for future performance

Our industry lies at a crossroads in balancing performance and power. By leveraging domain-specific processors and heterogeneous general-purpose computing, designers can optimize limited hardware resources and footprints. Optimizing designs and the design process across all types of multicore Systems-on-Chip (SoCs) also can achieve these gains.

While optimization might not get as much attention as multicore processing, it is equally important, especially in small-footprint applications with greater coherency challenges. Cache coherency is key to multicore computing applications, ensuring that the data stored in shared resources is properly maintained. Standards and specifications such as the AMBA 4 bus are encouraging steps toward providing system-level cache support across clusters of multicore processors, as well as maintaining prime performance and power efficiency in complex SoCs.

Future devices will continue to require more powerful processing performance, most likely under increasingly tight power constraints. By developing more targeted processing, optimization, and differentiation throughout the design process, developers can bring to market systems that not only support the many-core concept, but also incorporate software support.

John Goodacre is director of program management in ARM’s Processor Division. He has more than 20 years of experience in the engineering industry, including five years working for Microsoft as group program manager in the Exchange Server Group and as the manager of a team developing mobile phone software. John graduated from the University of York with a BSc in Computer Science.

ARM +44-1223-400-400 John.Goodacre@arm.com www.linkedin.com/company/arm www.facebook.com/ARMfans?ref=ts @ARMEmbedded www.arm.com

Whether software developers like it or not, and whether they’re prepared for it or not, virtually every semiconductor maker worth their salt is producing multicore Systems-on-Chip (SoCs). These SoCs typically pair two or more CPU cores with additional application-specific hardware accelerators to provide a complete system. For example, Cavium Networks, NetLogic Microsystems, and Freescale Semiconductor produce SoCs for network processing, while Texas Instruments and Broadcom make SoCs for digital media devices.

For software folks, this presents the interesting challenge of enabling applications to obtain all the processing power available from these multicore SoC environments. How can developers make sure their applications scale linearly with the available cores, as well as fully utilize the other SoC hardware components such as media accelerators and packet engines? To be clear, the scalability question is still a real science project for many applications; however, there are systems to build and products to ship, so developers can’t wait for the theoretically perfect solution.

In the past year, MontaVista Software examined numerous customer use cases in a wide range of applications including network processing, digital TV, in-vehicle infotainment, super low-power server Web hosting, and more. The goal was to understand how a Linux-based software solution could make full use of the underlying SoC hardware across a wide range of application requirements. The study identified the following high-level requirements that any solution must meet.

Multicore support

The demands of modern embedded systems are hastening the adoption of multicore SoCs. These demands are further accentuated by the requirements to run multiple systems simultaneously; thus, the solution must provide an efficient way of using and managing multicore environments.

Security

Anything downloaded to a device is insecure by definition. The solution must effectively isolate anything downloaded from the core device functions, and the downloaded applications must not be allowed to contaminate other applications.

Resource congestion

Downloaded applications must be prevented from hogging system resources. The goal is to effectively share resources such as memory, CPU time, and I/O. This sharing must allow more important system functions to have priority over less important downloaded applications.

Foreign system integration

Many environments run on top of a Linux kernel. However, these environments might require different userland libraries, as well as different kernel patches. For example, the Android system has its own device drivers and kernel patches. Ideally, the system could run any userland that runs on a Linux kernel. The kernel patches and userlands associated with these environments must be integrated with security and resource sharing in mind.

This analysis led to the development of a Linux-based architecture that maximizes the underlying power of today’s powerful multicore SoCs.

Architecture overview

To understand the overall architecture of this software, it is necessary to know a bit about modern Operating System (OS) environments, most notably virtualization technology. But be careful; there’s a lot of hype around virtualization (or, as we like to say, a lot of hype around hypervisors).

Virtualization is a method for dividing a computer’s resources into multiple execution environments. There are three major categories of virtualization in use today, with the key difference among them being the layer where virtualization occurs:

• Full virtualization and paravirtualization: These types of virtualization are used to host multiple guest OSs that are isolated from one another. While highly functional, the performance (without a great deal of optimization) is very low due to the overhead of the hypervisor and multiple OSs. Examples include QEMU, Kernel-based Virtual Machine (KVM), Zen, and VMware.
• OS resource virtualization: This type of virtualization is used to isolate and scale applications using a single OS. The advantage here is a single OS and lower overhead, typically less than 1 percent in most cases. Because there is so little overhead, the ability to scale and/or optimize performance is a huge benefit. Examples include Linux Containers and BDS Jails.
• Hardware segmentation (Asymmetric Multi-Processing or AMP): This  high-performance configuration dedicates hardware to specific applications running in user mode for maximum performance. This can be achieved using a simple runtime executive or leveraging OS resource virtualization and processor core affinity capability to dedicate cores and I/O to processes with almost no overhead.

These types of virtualization offer different performance characteristics, require different setup and maintenance overhead, introduce unique levels of complexity into the runtime environment, and address different problems.

While the industry is currently focused on pushing fully virtualized hypervisors as the one-size-fits-all solution to multicore optimization, the reality is that embedded developers need a range of options that can be tailored to specific application needs. Developers will require some combination of one or more of these virtualization technologies to deliver products that fit within hardware constraints and meet design performance characteristics. In short, the trick is to match the application with the right OS services to meet the overall system requirements, which can include performance, reliability, and security.

MontaVista provides three methods of virtualization based on nonproprietary, open-source Linux technology and supported across multiple processor architectures. Because it is a single runtime, there is one compiler and one set of tools that can be used for any use case or combination of use cases. Figure 1 shows an overall picture of this approach. These three methods are:

• KVM Hypervisor (full virtualization)
• Linux Containers (OS resource virtualization)
• MontaVista Bare Metal Engine (OS resource virtualization and SoC hardware segmentation)

(click graphic to zoom by 1.9x)

Microserver use case

The idea behind microservers is to utilize smaller, more energy-efficient processors to lower the physical and energy consumption footprint of a class of Web-centric IT applications. For certain workloads, several low-power processors can be more efficient than fewer, more powerful processors. Cavium Octeon processors and those from other semiconductor suppliers are well-suited to meet the density and power efficiency requirements underlying the microserver concept on the basis of the power efficiency of the cores themselves. These SoCs also include dedicated hardware to handle the front-end security and encryption/decryption processing that Web-based applications require.

From a software perspective, MontaVista Linux Containers and Bare Metal Engine technology help complete the picture. Containers are used to provide OS-level virtualization, allowing very efficient virtualization of the workload requirements. For example, Containers can be used to host thousands of independent websites, each securely isolated from each other. Containers allow the precise control of runtime resources allocated to each container, so each website can be limited to the performance levels the customer has purchased. Or, more importantly, a rogue website can be stopped from over consuming resources using the same mechanisms, thus thwarting a denial-of-service type of attack.

Bare Metal Engine provides the runtime environment for the security and encryption/decryption operations each of these hosted websites requires. For example, a 32-core SoC can utilize most of the cores for application processing with a few dedicated to packet processing, all controlled by one Linux instance.

Linux offers a simple solution

It is a widely held misconception that a combination of Linux and either a Real-Time Operating System (RTOS) or simple runtime environment must be utilized to fully realize the high performance available with multicore processors. Fueling this misconception is the thought that Linux itself is incapable of meeting the requirements because it is too big, too slow, and not real-time. This fallacy also drives the requirement that hypervisors and/or virtualization must mediate and isolate the different runtime environments and facilitate intercommunication among them. Often it is the RTOS vendors themselves who perpetuate this erroneous belief.

In the end, these misconceptions about Linux drive added complexity and costs into the development process. Complexity increases due to multiple runtime and development environments (one each for Linux, the RTOS, and possibly the hypervisor). Costs increase because of royalties for the proprietary RTOS and hypervisor, not to mention the added costs created by the development complexity itself, with more developers needed for a longer period of time.

The approach to use Linux everywhere and fix it where it might not meet some requirements results in a single OS environment, single tool chain, and common development and debugging tools for all aspects of the application. As Einstein said, “Make everything as simple as possible, but not simpler.”

(click graphic to zoom)

Jim Ready is the CTO of MontaVista Software and a recognized authority in the embedded systems and real-time software industry. The cofounder of Ready Systems, he developed the first commercially viable RTOS product: the VRTX real-time kernel. Jim invented the category of embedded Linux commercialization in 1999 when he founded MontaVista Software to provide the Linux OS to the embedded systems market and embedded system expertise to the open-source Linux community.

Patrick MacCartee is a director of product management at MontaVista Software in charge of hardware enablement, pricing, and channel strategies. Patrick has worked in high tech for more than 10 years at Intel and MontaVista Software. He is also responsible for managing the MontaVista Linux 6 and Carrier Grade Edition products.

MontaVista Software marketing@mvista.com @mvista www.mvista.com

ECD: What are the biggest challenges your customers talk to you about when looking to design automotive infotainment units?

GRYC: Automakers and Tier 1 automotive suppliers face many challenges, but implementing connectivity in mobile devices has become one of the most prevalent. Done right, mobile connectivity can help differentiate a vehicle and boost its desirability. But it also raises several issues:

• User satisfaction versus driver distraction: How do you design an infotainment system that can leverage the immense variety of apps and Internet services available on mobile devices while minimizing driver distraction?
• Time to market: How do you ensure that, by the time the car becomes available to the general public, its device connectivity isn’t already obsolete?
• Time in market: Given that an automobile’s life cycle is 10x longer than that of the average mobile device, how do you keep the car relevant? How do you ensure it continues to work with the latest mobile apps and services?

Sykes: In addition to the very important challenges related to mobile connectivity, customers want to balance the right set of features with the following system parameters:

• Performance: Given the growing sophistication of in-vehicle connectivity, applications, and Human-Machine Interfaces (HMIs), how can the customer ensure that the system will have enough performance to support the features required?
• Cost: An infotainment platform might need to support several configurations to span across a broad portfolio of vehicles. How do you optimize the development and bill of materials cost of the platform while meeting the unique needs of each vehicle?
• Power consumption: How can the customer minimize power consumption to offset an ever-increasing number of vehicle engine control units and the hybrid vehicle trend?

ECD: As the market matures and end-user expectations grow, user interfaces in automotive infotainment systems are becoming richer and more graphics-intensive. What’s being done to keep up with these requirements?

Gryc: Infotainment system designers are tapping into multiple standards and technologies, including Adobe AIR, HTML5, and OpenGL ES. Problem is, none of these can address every requirement. Thus, we provide a “universal platform” that supports these various technologies simultaneously. That way, designers can take a best-of-breed approach, blending apps and user interface components based on these technologies, all on a single display. It comes down to flexibility; rather than force our customers into an either/or approach, with all its attendant risks, we have opted for a both/and model that accommodates multiple environments.

Sykes: Graphical user interfaces in the vehicle are becoming more popular. As consumer devices such as tablets are introduced with ever-increasing graphical user interface capabilities, consumers expect the same experience in their vehicles. Freescale implements the latest graphics and multimedia technologies into our products, such as multiformat 1080p video encoding/decoding and high-performance graphics processing units with the latest graphics API standards such as OpenGL/GL ES, OpenVG, and OpenCL, and tests the products’ ability to withstand the harsh automotive environment for many years. In addition, Freescale works with ecosystem partners like QNX Software Systems to ensure that the software running on our products can take advantage of the hardware capabilities.

ECD: With today’s car owners desiring to bring their connected lifestyle to the automobile, what are the challenges to developing systems that easily integrate with smartphones and tablets? How are Freescale and QNX Software Systems addressing these challenges?

Gryc: Automakers need to concentrate on designing upgradable, future-proof systems that keep pace with the rapid evolution of mobile devices and applications. More specifically, they must create a system that can reliably and securely support new or upgraded software without affecting the system’s core functions. With this in mind, we equipped the QNX CAR Application Platform with several capabilities, including:

• Firewall and safety features, such as advanced memory protection to contain faults and adaptive time partitioning to prevent new applications from starving core processes of CPU time.
• A highly modular architecture that simplifies software updates using Firmware Over The Air (FOTA) techniques (see Figure 1).

(click graphic to zoom by 1.9x)

Sykes: Mobile devices are integrated into the vehicle environment in various ways. For example, an iPhone or iPad might require a USB interface, while many smartphones use a Bluetooth interface. This integration poses many challenges, including:

• How to account for future devices or software upgrades for existing devices.
• How to protect the in-vehicle system from corruption or viruses while interfacing to portable devices.

Freescale solutions offer the hardware and foundation software required for interfacing to a multitude of existing and future mobile devices, whether they utilize a wired or wireless interface. However, the ability to interface is only part of the challenge. It is also important to have available processing power to support future needs, which could include connections to other cars or to the transportation infrastructure. Once the interface is established, it is also necessary to ensure that it is secure. The i.MX processor family offers features such as specialized hardware for software version control and trusted image execution that can help protect the system from undesired software effects.

ECD: What are some of the key innovations both companies offer to simplify the design process for auto engineers? And how do you help the general application development community keep cars fresh with new applications?

Gryc: For auto engineers, it’s about creating a platform that has all the key software technologies integrated into it. In essence, we give them an out-of-the-box infotainment system that has the pieces they need – Operating System (OS) platform, multimedia engine, graphics frameworks, networking stacks, and so on – to get their systems set up quickly.

For the application developer community, it’s about providing support for HTML5, Adobe AIR, and other standards so that developers have the flexibility to apply their expertise to automotive systems, regardless of the application or the auto manufacturer for which the application is designed. For example, because the BlackBerry PlayBook is based on the QNX Neutrino OS, developers who create applications for the BlackBerry PlayBook could have a natural migration path between application development on the PlayBook and automotive systems based on QNX Neutrino.

Sykes: It is also important to provide scalable development solutions to the customer. While some chipmakers focus their solutions on a particular segment of the infotainment market, the newest i.MX 6 family of application processors can cover the entire range of solutions from entry to high end while maintaining software compatibility (see Figure 2). The entire family is based on standard ARM processors in single and multicore configurations and shares common footprints and pin configurations. Hardware can be designed without worrying if more or less performance will be needed later.

(click graphic to zoom by 1.9x)

Prior to hardware development, the software development process can start with Freescale’s affordable automotive-focused reference platforms that enable key system functions, offer expandability to support new functions, and include the software base that allows the customer to start development right away.

ECD: What has changed in automotive infotainment that makes multicore CPU architectures so attractive to developers today, and how do Freescale and QNX Software Systems take advantage of these architectures?

Gryc: Simply put, there’s a lot more going into cars, from video codecs to 3D navigation applications to high-level languages with virtualized execution environments, all of which have a serious appetite for CPU cycles.

Multicore addresses this demand for greater processing power by allowing multiple applications to run in parallel. For example, in a multimedia head unit, you could dedicate one core to running a compute-intensive process such as an HTML5 browser and use the other cores to run the remaining processes. Or you could run in full Symmetric Multi-Processing (SMP) mode and allow any process to run on any available core.

From a software perspective, the QNX Neutrino RTOS makes multicore dead simple. Because QNX Neutrino employs a true microkernel architecture, only the OS kernel needs SMP awareness and supporting logic. Applications, drivers, networking stacks, and other multithreaded processes can automatically take advantage of multiple cores without having to be rewritten or redesigned because the kernel handles the details of scheduling threads on each core. Moreover, our visualization tools can analyze how the multicore system behaves as a whole. This approach offers deeper insight into the complex system interactions typical of multicore designs and allows developers to focus their efforts on areas that yield the greatest increase in parallelism and performance.

Sykes: Multicore CPU solutions offer the benefits that Andy described while consuming less power than a single core CPU with similar performance. This reduction in power consumption is important for all vehicles, but can be critical in hybrids.

While some markets focus on achieving the highest performance for a particular application, infotainment applications can greatly benefit from multicore CPUs because they require multiple simultaneous processes such as speech, audio, multimedia, navigation, and HMI. Single applications running across multiple cores will experience diminishing returns as the number of cores increases. In contrast, multiple applications in an infotainment system can be run as unique processes to optimize loading and take full advantage of each core’s available performance. As a result, the infotainment system achieves more processing power with better software load balancing, as well as lower power consumption.

(click graphic to zoom)

Andy Gryc is the automotive product marketing manager at QNX Software Systems. He has been a software developer and designer for more than 20 years. Prior to joining QNX Software Systems, he worked as the lead embedded software architect for GM OnStar, designed and implemented a speech recognition engine, and worked on software for palmtops and notebooks.

Paul Sykes is a marketing manager with Freescale’s Driver Information Systems organization. He has more than 15 years experience in the semiconductor industry, including product development, program management, and marketing positions. He is currently focused on the automotive telematics, infotainment, and driver information application spaces.

QNX Software Systems 800-676-0566 info@qnx.com www.facebook.com/pages/QNX-Software-Systems/92022217198 @QNX_News, @QNX_Paul www.qnx.com

Freescale Semiconductor www.facebook.com/freescale @Freescale www.freescale.com

Embedded virtualization has several positive implications for OEMs. For example, once there is a means for splitting up applications to run on multiple cores while maintaining determinism, the solution can subsequently enable real-time applications to scale the number of cores they use, upward or downward. With scalability, OEMs can offer a range of price/performance options for their products without requiring changes to the software.

Virtualization is not a new concept in computer science, but it has gathered new interest with the advent of multicore processors. Though virtualization is recognized as a way to keep multiple processor cores busy, it’s important to note that most types of server or client virtualization are not designed to meet the needs of time-critical embedded processing. These approaches to virtualization most often treat all processors on a multicore chip the same way. In these systems, a single Operating System (OS) assigns tasks to processors as they become available in an attempt to keep all processors as heavily loaded as possible with processing tasks.

Server and client virtualization typically virtualize all hardware including the I/O interfaces. When an I/O interface needs servicing, the Virtual Machine Monitor (VMM) fields the request and passes the results to the OS clients it is supporting. There is no way to ensure that a particular OS client is loaded when an I/O that belongs to that client requires service, nor is there a global way of associating a particular I/O with a client OS and the application running on it. Consequently, there is no way to guarantee exactly how much time will be required to handle an I/O event; hence, this approach is not appropriate for handling real-time processing in an embedded system.

Embedded system designers want direct control of the system to obtain determinism and consistent performance. While it is desirable to balance overall processor utilization and keep multicore processors as busy as possible, this is not the top priority. First and foremost, embedded designers are looking for software technology that helps them maintain determinism while adding features and/or reducing the cost of their OEM products.

Embedded designers are looking for a software platform that enables them to combine OSs of different types so that processing can be optimized for the tasks at hand – for example, real-time OSs to handle critical I/O timing requirements and General-Purpose OSs (GPOSs) to leverage COTS graphics-rich applications that run human-directed functions. They are also looking for solutions to scale applications so they can provide different products using the same application code base. This reduces engineering development cost, improves time to market, and more importantly, enables new products to be based on tried and proven software that can be continuously upgraded in performance and reliability.

Embedded virtualization preserves determinism

For an embedded application to be deterministic, it must be engineered as such from the inception of the development project. Determinism is not something that can be added at the end. Special considerations must be made to ensure that application threads have direct control of the I/O interfaces on which they depend.

Figure 1 shows a pick-and-place assembly system in which TenAsys’ INtime for Windows Real-Time Operating Systems (RTOSs) are hosted on three cores of a quad-core processor and the Human Machine Interface (HMI) is hosted on the fourth core running Microsoft Windows. The real-time tasks running on different CPUs communicate when required via the global object network. This automated assembly system includes three real-time subsystems: a vision system that guides an assembly robot, the multi-axis robot, and the material transport system that indexes components into place for assembly and then carries off assembled units. The ideal way to develop and debug this type of application to ensure that each component performs as reliably as required is to split the application into separate components.

(click graphic to zoom)

For example, the HMI would be a separate application module that might run in a non-real-time environment such as Windows. This requires an OS environment that can partition the platform resources – I/O, memory, interrupts, and CPU core (in the case of a multicore processor platform) – to allow the application modules to run totally independently from each other. An embedded virtualization environment supports this, allowing different real-time tasks to run on specific processor cores. The operating software partitions physical I/O interfaces so that an interrupt coming from one of the devices only interrupts the processor handling that device. This ensures predictable response times for handling real-time events.

The hosting of multiple OS environments on a multicore chip is managed either by software called an embedded virtualization manager or by special embedded virtualization functionality contained in the RTOS.

Embedded virtualization can be implemented in different ways, depending on the amount of hardware virtualization support provided by the processor. Paravirtualization solutions use software techniques to modify the guest OSs, allowing them to work side by side with-out affecting each other or compromising the system’s real-time responsiveness. Implementations provide varying degrees of platform partitioning and have typically been limited to running two OSs at a time on a platform – an RTOS and a GPOS. Some implementations have evolved to the point where the GPOS doesn’t require any modification and the latest version of the GPOS is readily supported. This is a real plus when the object of coupling the GPOS to the RTOS is to make use of legacy RTOS application software as is, without any modification, while adding an HMI based on an OS like Windows that leverages the latest HMI development tools.

Over the years some of these implementations have been optimized to provide the best performance for a particular combination of OSs. The downside to this is that each implementation is specific to the particular combination of OSs, and it is an impractical approach to providing a generic virtualization solution to support multiple OS combinations.

Hardware-assisted virtualization features like VT (supported by Intel processors) that have recently become available eliminate some of the software complexity of paravirtualization by providing hardware assistance built into the processor. By using the hardware virtualization support, a VMM can be constructed to function without knowledge of the guest OS. As a result, the VMM can support any OS targeted for that platform.

Intel processor features like VT-x (a subset of the VT features) ensure that any memory address issued by a guest OS is automatically mapped to the appropriate address location in physical memory. Likewise, the hardware-assisted virtualization feature called VT-d automatically maps I/O memory accesses for bus-master DMA devices, enabling native I/O drivers that are part of the guest RTOS application to be used without modification in the virtualized environment. These hardware-assist features substantially reduce the complexity of a VMM and make embedded virtualization a more viable solution.

Making scalability work in real-time applications

While embedded virtualization provides the ideal environment for enabling real-time applications to be split into individual independently operating components, breaking up applications creates the need for a mechanism to support Inter-Process Communications (IPC).

In the past, designers often set up Ethernet links between application subsystems and used TCP/IP stacks to communicate between the subsystems, but this method is cumbersome, slow, sometimes unreliable, and adds uncertainty to the system’s behavior, affecting determinism.

A better IPC approach is to use a concept called global object networking. A global object network provides a managed communication environment with built-in initiation and discovery services, enabling an application to be dynamically distributed across one or several CPUs at load time. Processes requiring services of other processes are found automatically, and a local manager records their location to keep track of established IPC links. If a communications link or targeted process fails, the manager notifies an initiating process. In addition, the local manager keeps the system clean by clearing up all records when the IPC links are no longer required by the initiating process. Because the global object network is integrated with the OS, its overheads are low and it does not require the application developer to create any custom software. It is deterministic and substantially more efficient than traditional IPC interfaces.

The OS manages the location and existence of global objects through which processes pass information to ensure the system’s integrity. For instance, objects used by a process across several processors are “kept alive,” and only removed when all the processes have terminated. This requires an underlying management infrastructure to ensure that an object is not removed prematurely, or that it isn’t removed causing memory leakage through poor cleanup of unused resources. Likewise, in the event that a processing node goes down before its processes are terminated, the manager needs to inform the global object managers on all the other processing nodes that they should clean up any local references to objects on the down node.

An example of GOBSnet communications

Figure 2 shows a highly simplified view of the software architecture of the system depicted in Figure 1. In addition to the RTOS and the real-time process software, Cores 0-2 also run the INtime GOBS manager software, whose function is to manage global object communications. (GOBSnet is TenAsys’ global object network supported by the company’s INtime Distributed RTOS and INtime for Windows RTOS.)

(click graphic to zoom by 1.9x)

With GOBSnet, a process on one core can communicate with another process on another core using a global memory object. The initiating process, “Process 1” in Figure 2, creates the memory object and catalogs it in that core’s root process. After this is done, processes running on other processing nodes can find the memory object, resulting in an efficient shared memory interface for all processes to use. The same applies to all ranges of objects (including semaphores and mailboxes) for IPC use.

The second step is for other processes to locate the memory object and obtain its memory location. This is done by specifying the processor name to start the search. When the process finds the object, it stores its location, type, and parameter in a reference object in the processor node’s GOBS manager and keeps the handle of that reference object. From then on, when a remote process (Process 2 or Process 3 in this example) wants to write or read to the memory object, it uses the reference object’s handle to retrieve the appropriate memory object information to access it.

When Process 2 or Process 3 terminates its node’s GOBS manager, the manager clears all reference object information about the remote memory object. When Process 1 terminates, it removes all objects it has created, including the memory object. In cases where that is not the optimum action, there is the option of creating a memory object with a counter. The counter is incremented every time another process connects to the memory object and is decremented every time one of those associated processes is terminated. The result is that the memory object is removed only when all the processes connected to it are terminated. This allows the situation where remote processes, Process 2 and Process 3, can continue passing information via the memory object even though the initiating process, Process 1, has terminated.

GOBSnet communications can be used whether the real-time application is spread among CPUs on the same multicore chip or among separate CPU cores on different microprocessor components that are networked together (Figure 3). By designing systems around this flexible system software architecture, embedded system developers have headroom to grow their products’ processing power or shrink it accordingly to meet the challenges of the future.

(click graphic to zoom)

Kim Hartman is VP of TenAsys Corporation. He has worked in the embedded market focusing on hardware analysis tools and RTOS products for 26 years, first at Tektronix and then at RadiSys before cofounding TenAsys in 2000. He is a Computer Engineering graduate of the University of Illinois, Urbana-Champaign, and received his MBA from Northern Illinois University.

TenAsys Corporation 503-748-4720 Kim.Hartman@tenasys.com www.tenasys.com

The key performance metric of interest for military DSP systems is the speed of performing floating arithmetic operations, which is referred to typically as GFLOPS when discussing the speed of computers. In recent history, these DSP systems were commonly built using Texas Instruments 320C40 and 320C6701k and Analog Devices SHARC dedicated DSP processors, which were themselves followed by a number of generations of PowerPC/Power Architecture processors with AltiVec. All of these processors offered good floating point performance per watt and all were available from vendors with a history and track record of support for military embedded customers. Now, with the introduction of Intel’s 2nd Generation Core i7-2715QE quad-core processor, the design of x86-based embedded military DSP systems and high-performance SBCs takes a significant leap forward.

Intel refers to their product introduction cadence as the “Tick-Tock” model. A “tick” is when Intel delivers new silicon process technology with increased transistor density, and enhanced performance and energy efficiency within a smaller version of an existing microarchitecture. The 2nd Generation Intel Core i7 is a “tock,” which is when an entirely new microarchitecture is introduced on an existing semiconductor process technolgoy. Using the 32 nm process introduced with the Westmere generation, the 2nd Generation Core i7 (previously code-named “Sandy Bridge”) features many architectural improvements (especially in the cache subsystem) that lead to improved performance per clock cycle. It is the nature of microprocessor design that revised architectures typically provide incremental performance improvements. However, the 2nd Generation Core i7 has delivered a major leap forward in the signal processing capability of the processor, thanks to the new 256-bit wide Intel Advanced Vector Extensions (AVX) floating-point instruction set, which supercedes the earlier 128-bit Streaming SIMD Extensions (SSE) instructions.

While the new Core i7 brings many advantages for DSP system designs, SBCs used in conjunction with Core i7-based DSP engines also benefit. SBCs can now take advantage of the first ever support for Serial RapidIO on Intel Architecture, as the result of an upcoming PCIe2-to-Serial RapidIO2 bridge chip from IDT that will provide a common communications path and improve interoperability in a complete system. The new Intel processor also supports 16 lanes of Gen2 PCIe for full-bandwidth communications across high-performance processor cards. Intel’s hyperthreading technology provides for running two execution threads on each core, enabling greater utilization of the execution units and providing improved power efficiency. Published reports show performance increases of 7 to 34 percent due to hyperthreading alone.

The AVX 256-bit difference

Prior to the introduction of Intel’s new 256-bit AVX, developers of military DSP systems typically turned to 128-bit AltiVec-enabled CPUs for vectorized signal processing functions. In the past few years, development of new AltiVec-enabled processors slowed significantly, leaving DSP system developers with limited options. In the meantime, Intel continued to invest in and enhance its own high-performance vectorized processing solution with continual enhancements to Intel Streaming SIMD Extensions, a 128-bit wide processing unit predecessor to AVX, capable of simultaneously operating on four 32-bit floating point values. Intel SSE also featured support for double-precision floating point, a feature not available in AltiVec. In Intel’s earlier multicore processors, each core was provided with its own SSE unit, so raw floating-point performance scaled with the number of cores. In the new Core i7 Intel has upgraded SSE with AVX, doubling the size to 256-bits wide.

This doubled vector processing performance is a significant milestone in DSP system design. DSP algorithms used in critical military applications such as radar, SIGINT, and image processing depend on the precision achieved with floating point numbers combined with the speed of processing. The new Core i7 doubles the peak performance of SSE. When compared to SSE in actual FFT kernels, AVX has been benchmarked up to 1.8x faster than SSE (Figure 1). The AVX instruction set was designed to support future extensions, which hints at wider implementations in the future.

Figure 1: A comparison of complex FFT performance using 128-bit SSE versus 256-bit AVX instructions – The code was run on the same processor and clock rate. Data was provided by Intel.

Serial RapidIO onboard

Serial RapidIO is the preferred fabric for the types of processor-to-processor communications required by demanding military DSP systems. This is because of Serial RapidIO’s reliable packet transmission and ability to deliver low and predictable latencies. These benefits of RapidIO messaging are ideal for large peer-to-peer clusters of processors typically used in complex signal processing applications. With the Intel 2nd Generation Core i7, Serial RapidIO is supported on Intel architecture-based OpenVPX/VITA 65 embedded boards for the first time with an easy, cost-effective interconnect provided by IDT’s upcoming PCI Express (PCIe) Gen2-to-Serial RapidIO protocol conversion bridging semiconductor product.

Before this newest generation of Core i7, the lack of support for Serial RapidIO for Intel platforms severely limited the viability of using Intel architecture in DSP multiprocessor system designs. Solutions for Intel have included support for fabrics such as InfiniBand and Gigabit/10 Gigabit Ethernet, which are not embraced in military applications because of their non-industrial temperature silicon and relatively high power consumption. For SBCs, where the requirement is typically a single processor communicating with I/O, these fabrics have been sufficient, but would-be Intel-based DSP military designers were deprived of the option to design systems around Serial RapidIO, the multiprocessor fabric of choice.

IDT’s PCIe-to-Serial RapidIO bridge and new Gen2 Serial RapidIO switches will enable system designers to build Intel architecture-based processing engines with much more fabric bandwidth than that offered by any other currently available technology. The upcoming IDT bridge product supports 5 Gbps interfaces on both PCIe and Serial RapidIO ports. With the advantage of small size and low power consumption, system designers can add bandwidth by using multiple PCIe2-to-Serial RapidIO2 bridges connected directly to the processors or via a PCIe switch. This performance can scale at the system level with the new Gen2 Serial RapidIO: This new generation of systems will deliver double the backplane bandwidth provided by the already fast 3.125 Gbps Gen1 Serial RapidIO technology. A 19" rack, OpenVPX processing system will be able to deploy 1.2 terabits per second of fabric bandwidth. The Intel/Serial RapidIO combination is also suited for SwaP-constrained systems, as designers can maximize the power available for actual computing knowing that Serial RapidIO fabric technology provides the best bandwidth/watt.

Serial RapidIO bridges implemented in FPGAs don’t support high-performance messaging, a feature which directly maps to higher-level software APIs such as MPI. IDT’s new bridge product will support the two main Serial RapidIO transfer modes, Serial RapidIO messaging, and memory-mapped transfers. Another benefit of the IDT silicon is the inclusion of DMA engines that speed computation while offloading the host processor. Intel processors typically don’t have DMA engines on-chip, but depend instead on the peripheral chip to move data. Without a DMA engine, moving data can require a large amount of the host processor’s attention, with the result that a multicore processor might have one of its cores (and associated power) largely consumed by moving data, which is all the more burdensome because it has to be done in code.

Another advantage of Serial RapidIO for SWaP-constrained military systems is its ability to support distributed switch and centralized switch architectures. Distributed switch systems (an example is the VITA 65 BPK6-CEN05-11.2.5-n backplane profile) can make use of the local Serial RapidIO switch and thus avoid the need for a separate switch card. For example, if the system were using a ½ ATR Short enclosure (four 1" slots), this capability saves 25 percent of the space and a considerable amount of power. For large systems, centralized switch architectures are often preferred, and Serial RapidIO is equally adept at this approach.

An example of a high-performance DSP engine designed to take full advantage of the latest offering for Intel’s Core i7 is the new CHAMP-AV8 from Curtiss-Wright Controls Embedded Computing (Figure 2). The CHAMP-AV8 is an Intel Core i7-2715QE-based rugged, high-performance OpenVPX DSP engine. Performance of this dual Core i7 DSP engine is rated at up to 269 GFLOPS. It also supports the IDT Gen2 PCIe-to-Serial RapidIO bridge product, effectively tripling the bandwidth of first-generation VPX products with up to 240 Gbps of fabric performance. CS

Figure 2: The Curtiss-Wright CHAMP-AV8 VPX board with 269 GFLOPS and featuring dual Intel Core i7 2715QE CPUs with four Gen2 Serial RapidIO ports and 80 Gbps to the backplane. (click graphic to zoom by 1.9x)

Ian Stalker is the DSP product manager for Curtiss-Wright Controls Embedded Computing. He has more than 20 years of experience in the embedded industry and holds a degree in Electronics Engineering. Contact him at ian.stalker@curtisswright.com.

Alan Baldus, Intel SBC Product Marketing Manager at Curtiss-Wright Embedded Controls, has more than 15 years in the embedded computer industry. He holds a BS in Electrical Engineering. Contact him at abaldus@curtisswright.com.

Curtiss-Wright Controls Embedded Computing

703-779-7800

www.cwcembedded.com