National Corporation for Atmospheric Research

The National Center for Atmospheric Research (NCAR) is a federally funded research and development center devoted to service, research and education in the atmospheric and related sciences. NCAR’s mission is to understand the behavior of the atmosphere and related physical, biological and social systems; to support, enhance and extend the capabilities of the university community and the broader scientific community – nationally and internationally; and to foster transfer of knowledge and technology for the betterment of life on Earth. The National Science Foundation is NCAR’s primary sponsor, with significant additional support provided by other U.S. government agencies, other national governments and the private sector.

Business situation
In its development of the 4-D Wx Data Cube, NNEW is creating two pieces of software that showcase how the OGC-defined Web Coverage Service (WCS) and Web Feature Service (WFS) standards can be implemented. These Reference Implementations serve as both proofs-of-concept (i.e., prototypical implementations of the WCS and WFS standards) and standards in and of themselves, which outside organizations can take and improve upon. They are de facto feature roadmaps, describing through example what future implementations of the WCS and WFS standards should include in order to be compatible with the Cube.

Technical situation
The original services provided by NCAR were running in a stand-alone servlet container and were not providing clustering, offline work, delegation nor separation of concerns. The initial focus was on bringing these services into a computing center worthy status, providing a scalable asynchronous model for publish-subscribe as well as re-using SOA techniques and freely available open source components and tools.

The core platform for deployment is based on OSGi, the container use is depending on configuration Apache Felix or Eclipse Equinox. The container framework is utilizing Apache ServiceMix and Apache Karaf to provide a runtime environment suitable for EE and SOA applications. In it’s first iterations the WCSRI reference implementation was built around Spring-DM, it later was refactored to a full Apache Aries Blueprint implementation using the core, JPA, JNDI and Transaction modules from the Apache Aries project; the main reasoning behind this was operational stability, scalability as well as interaction with the OSGi container. The WCSRI implementation also provides a modular database abstraction layer, tested against the most common commercial and open source relational database offerings, a complete web-services unifier and clustering components to handle heartbeat, concurrency as well as regional data centers. The final solution is also fully XA aware and relying on container managed transactions across routing, web-services and messaging as well as persistence layers. The implementation of WCSRI also led to several bug-fixes and contributions back to the open source projects used to build the final solution.

WCSRI has since its inception positioned itself extremely well in FAA tests, achieved publish subscribe metrics well beyond the initial demands placed on the solution as well as becoming a reference for how to design publish subscribe models with extremely large payloads in a clustered environment. In its current release the WCSRI system is:

  • Using established Java Specs
  • Conforming to OSGi standards work
  • Relying on simple to use and migrate messaging specs
  • JAX-WS and JAX-RS based
  • Completely modular
  • Has removed all payload stream issues from memory constraints

Products and services company used

The components utilized in the WCSRI system:

  • Apache ServiceMix
  • Apache Camel
  • Apache CXF
  • Apache ActiveMQ
  • Apache Felix
  • Apache Aries (Blueprint, Transaction, JPA, JNDI)
  • Apache OpenJPA
  • Apache Derby
  • Oracle RDBMS
  • Postgresql
  • MySQL
  • Eclipse Equinox
  • Google Guice