The National Center for Atmospheric Research (NCAR) is a federally funded research and development center devoted to service, research and education in the atmospheric and related sciences. NCAR’s mission is to understand the behavior of the atmosphere and related physical, biological and social systems; to support, enhance and extend the capabilities of the university community and the broader scientific community – nationally and internationally; and to foster transfer of knowledge and technology for the betterment of life on Earth. The National Science Foundation is NCAR’s primary sponsor, with significant additional support provided by other U.S. government agencies, other national governments and the private sector.
In its development of the 4-D Wx Data Cube, NNEW is creating two pieces of software that showcase how the OGC-defined Web Coverage Service (WCS) and Web Feature Service (WFS) standards can be implemented. These Reference Implementations serve as both proofs-of-concept (i.e., prototypical implementations of the WCS and WFS standards) and standards in and of themselves, which outside organizations can take and improve upon. They are de facto feature roadmaps, describing through example what future implementations of the WCS and WFS standards should include in order to be compatible with the Cube.
The original services provided by NCAR were running in a stand-alone servlet container and were not providing clustering, offline work, delegation nor separation of concerns. The initial focus was on bringing these services into a computing center worthy status, providing a scalable asynchronous model for publish-subscribe as well as re-using SOA techniques and freely available open source components and tools.
The core platform for deployment is based on OSGi, the container use is depending on configuration Apache Felix or Eclipse Equinox. The container framework is utilizing Apache ServiceMix and Apache Karaf to provide a runtime environment suitable for EE and SOA applications. In it’s first iterations the WCSRI reference implementation was built around Spring-DM, it later was refactored to a full Apache Aries Blueprint implementation using the core, JPA, JNDI and Transaction modules from the Apache Aries project; the main reasoning behind this was operational stability, scalability as well as interaction with the OSGi container. The WCSRI implementation also provides a modular database abstraction layer, tested against the most common commercial and open source relational database offerings, a complete web-services unifier and clustering components to handle heartbeat, concurrency as well as regional data centers. The final solution is also fully XA aware and relying on container managed transactions across routing, web-services and messaging as well as persistence layers. The implementation of WCSRI also led to several bug-fixes and contributions back to the open source projects used to build the final solution.
WCSRI has since its inception positioned itself extremely well in FAA tests, achieved publish subscribe metrics well beyond the initial demands placed on the solution as well as becoming a reference for how to design publish subscribe models with extremely large payloads in a clustered environment. In its current release the WCSRI system is:
- Using established Java Specs
- Conforming to OSGi standards work
- Relying on simple to use and migrate messaging specs
- JAX-WS and JAX-RS based
- Completely modular
- Has removed all payload stream issues from memory constraints
Products and services company used
The components utilized in the WCSRI system:
- Apache ServiceMix
- Apache Camel
- Apache CXF
- Apache ActiveMQ
- Apache Felix
- Apache Aries (Blueprint, Transaction, JPA, JNDI)
- Apache OpenJPA
- Apache Derby
- Oracle RDBMS
- Eclipse Equinox
- Google Guice
Melbourne IT is an internet service provider with a large footprint in southwestern Australia. They provide internet services as well as domain services and integrate with other ISP’s worldwide to provide a large number of products to their customers. They had jumped into open source technologies and service oriented architecture (SOA) with both feet, but had little experience implementing or using these products and soon felt they were in over their head. They went to a consulting firm that claimed to be able to help. After 3 months and hundreds of thousands of dollars, they realized that this group had little more knowledge then their own staff. So they turned to Savoir Technologies.
We sent in a specialized consultant to meet their needs. He worked hand in hand with a small group of developers and architects from different departments within Melbourne IT in order to ensure all parties were on the same page. He looked at what was delivered by the other consulting firm to suggest changes and areas for improvements where code was inefficient or wrong. Our consultant was able to setup a new base project for them to build from and architect a new implementation that would make the system more scalable and highly available using ActiveMQ and ServiceMix. Savoir’s intimate knowledge of SOA and open source enabled Melbourne IT to boot strap their SOA initiatives using open source with best-in-breed development methodologies and architectures.
LeapFrog Enterprises Inc NYSE: LF is an educational toy company based in Emeryville, California. LeapFrog designs, develops and markets technology-based learning products and related content for the education of infant through grade school children at home and in schools internationally.
Leapfrog has been developing learning tools and toys for children for over 15 years. And with the onset of the internet, they saw an opportunity to develop and store data from each child that was gathered on the handhelds educational toys and learning devices. This would allow them to provide feedback to parents on the progress of their child and give awards to the children for completing tasks. This would also make the toys more marketable and allow flexibility with upgrades and downloadable updates.
Leapfrog needed to provide a complete services layer and enterprise service bus that allowed users to create accounts, login, connect and register devices, and monitor child progress. The initial roll out provided the basics, but was not nearly ready for the holiday season and the expectation of 1.5 million new accounts on Christmas Day. With the current services layer in place, account creation was taking approximately 1 minute and 30 seconds to complete. Each account creation required 2 round trips to an outside vender.
An open source services solution was put in place providing CXF endpoints for integration with the customer facing UI components. These endpoints were used by several front-end technologies to simplify the development and integration of all Leapfrog’s UI’s. The account creation implementation was reworked to change the source of accounts to be controlled by Leapfrog and allowing the services to asynchronously manage messaging from the front-end. This took the account creation time from one and half minutes to less then 1 second. Enterprise service bus was used to transform information asynchronously to make efficient use of CPU and server utilization. IO was analyzed to provide for aggregation to minimize the number of round trips from the web application to the backend systems and third party vendors. Effective use of Enterprise Integration Patterns provided best practice use of backend processes, leveraging additional efficient use of CPU and server resources.
With the new services layer in place, Leapfrog was able to handle the load of new customers on Christmas Day. The services layer was able to handle the largest load Leapfrog has ever experienced.