Seminar by Prof. Dr. Thomas STRANG

Applied Computer Science Problems”Basic problems of mobile services and their solutions”Seminar by Prof. Dr. Thomas STRANGWS 2004/2005Mobile Frameworks: Ninja & OSGIBernhard Leschinger Bernhard.Leschinger@uibk.ac.atJanuary 20051 INTRODUCTION (3)2 NINJA FRAMEWORK (3)2.1 NINJA Introduction (3)2.2 NINJA Concepts (4)2.3 NINJA Architecture (5)2.4 Programming Model and Design Patterns (7)2.5 vSpace Execution Platform (8)2.6 Distributed Data Structures (8)2.7 Service Discovery Service (SDS) (9)2.8 NINJA Example Services (9)3 OSGI FRAMEWORK (10)3.1 OSGI Overview (10)3.2 OSGI Introduction (10)3.3 Key Features of the Service Platform (12)3.4 OSGI Architecture (13)3.5 OSGI Services (13)4 CONCLUSION (15)5 APPENDIX (16)1 IntroductionThe seminar deals with basic problems of mobile services and their solutions. In this paper I will discuss the two mobile frameworks NINJA from the University of Berkeley and the OSGI framework from the OSGI Alliance, both trying to deal with the problems of mobile services. Some of these problems are: different clients with small displays, weak computing power, low memory, discontinued connection to the network and services, geographically moving devices, cheap devices that can be lost or broken and all data has to be reconstructed through the services. Another big issue are security concerns. Both frameworks try to deal with these problems in a different and unique way as I will describe in this paper.2 NINJA FrameworkThe Ninja Project was initiated in spring 1997 at the UC Berkeley Computer Science Division. Their homepage can be found at / .The idea of the NINJA project was to enable internet-scale services from arbitrarily small devices.2.1 NINJA IntroductionNINJA was developed by a team of the University of Berkeley on linux platform. NINJA software is a prototype and is in development status, and not complete. It was almost completely developed in Java with Java version JDK 1.1. Therefore the project is platform independent. In addition to that, the source code is freely available at the following web address: /software.htmlThe project wants to offer a software infrastructure for web/ and internet based services. Central to the NINJA approach is the concept of a service, which is an internet-accessible application (or a set of applications) which is:scalable (able to support many thousands of concurrent users)fault-tolerant (able to mask faults in the underlying server hardware)and highly-available (resilient to network and hardware outages)The NINJA project was tested with a few examples running at the Univeristy of Berkeley campus. These examples were:an Internet stock-trading systema ``universal inbox'' used to access e-mail, voice mail, pages, and other personal correspondenceand the Ninja Jukebox, which provides real-time streaming audio data served from a collection of music CDs scattered about the network2.2 NINJA ConceptsNINJA intends to build upon and expand the notion of web-based services by providingcomposability (the ability to automatically aggregate multiple services together into a single entity)customizability (the ability for users to inject code into the system to customize a service's behavior)and accessibility (the ability to access the service from a wide range of devices, including PCs, workstations, cellphones, and Personal Digital Assistants).The end goal of the NINJA project is to enable the development of a varied mixture of internet-based services which are interoperable and immediately accessible across the spectrum of user devices ranging from PCs and workstations to cellphones and personal digital assistants (PDAs).For example, one should be able to check one's e-mail simply by calling a special number from a cellphone, or equivalently by sitting down at any internet-connected PC in the world.The key ideas employed in NINJA which I want to explain in detail now are the following:Structured Partitioning of StateOperators and PathsAutomatic Service CompositionMobile Code for Service Deployment2.2.1 Structured Partitioning of StateDistributed, wide-area state management is hard. As such the Ninja architecture makes a clear distinction between hard, persistent state and soft state: hard state is maintained in a carefully-controlled environment, the Base, which is engineered to provide high availability and scalability.All other service state is soft and therefore can be regenerated or recovered in case of loss. For example, a client-side e-mail cache can always be regenerated from the data stored on the e-mail server.2.2.2 Operators and PathsAll service components are constructed as strongly-typed Operators which perform a specific operation on the data passing through them, such as transformation or aggregation.Operators can be manually or automatically composed into a chain of operations comprising a Path. The Path is the "session" upon which data flows between the service (or services, since more than one may be part of a Path) and the end user.2.2.3 Automatic Service CompositionTagging all Operators with appropriate metadata allows Paths to be automatically generated from a set of constraints, e.g., "build me a path from my calendar service to the simple text interface on my PDA". Such a Path might contain Operators to transform internal calendar data to a simple SGML format, which is then transformed to a simple UI for the PDA, and vice versa. Accessing the same calendar data from a cell phone would involve Operators to convert between voice and TTF signals and the calendar service interface. Each of these components can be built separately and composed on the fly by the Ninja infrastructure.2.2.4 Mobile Code for Service DeploymentApart from Bases and end user devices, Active Proxies form the middle tier, which tie the two together in the NINJA architecture.An Active Proxy can be thought of as an anonymous code-hosting site on the Internet, capable of running simple (or potentially complex) NINJA Operators which do not require the use of persistent state (that is, Active Proxies only manage soft state). Operators are instantiated on Active Proxies on demand, allowing portions of a Path to be geographically located to maximize performance, network bandwidth efficiency, and so forth.For example, it might make sense to perform service data caching at an Active Proxy close to the user, in case the Base hosting that data is geographically (or network-wise) distant.2.3 NINJA Architecture2.3.1 BASESBases are clusters of nodes, which are clusters of workstations. The core of every service has to run at a base. Every node does replication of computation and data to provide fault-tolerance on cluster nodes.A cluster permits incremental scalability as nodes can be added to increase capacity. Coupled with high performance system area networks, clusters can deliver excellent performance for relatively low cost.Designing software to run on clusters of workstations is known to be difficult. To simplify the task of authoring new services, NINJA offers a cluster-based software platform called “vSpace” that allows serviceauthors to concentrate on application specific functionality, rather than on details related to scalability, fault tolerance, and composability. Services designed with that vSpace inherit the essential service properties from the platform which greatly reduces the size and complexity of the service’s codeThe vSpace platform supports dynamic uploading of new services by either trusted or untrusted third parties. So authors can construct and test their services locally, but then upload their services into bases that are externally maintained.2.3.2 UNITSToday, more and more different devices are connected to the internet. In addition to PCs, laptops, PDAs and other mobile devices even sensors and actuators are connected. This large family of client devices, here called units, may have limited connectivity and low or intermittent bandwith. They also may have poor computational abilities, and may be able to handle only a small set of data formats and network protocols. Units may have a small display and no big memory to store data. They can be easily lost or broken and so any state they manage must be replicated in a durable environment such as a service running in a base. Inexpensive units may not have enough computational ability to handle the rich set of data types and the growing set of protocols – such units must rely on surrogates that adapt content and protocols on their behalf.Mobile units may experience regular periods of disconnected operation. Bases can assist such weakly connected units with the consistency management of data shared across units.2.3.3 ACTIVE PROXIESBetween devices and services there are so called “active proxies”. These are transformational intermediaries which transform data between services and devices. They can adapt protocols between the two, e.g. transform an cpu consuming SSL session into a less expensive shared key encrypted channel for CPU constrained devices. They can adapt content and filter (remove) sensitive information before content is shown on an untrusted access point.Active proxies bring three essential properties to the NINJA platform:Dynamic Service AdaptationNetwork protocols and data formats are translated between clients and services to cope with the different units using the network. (e.g. HTML->WML for WAP) They adapt data for the clients with low computing power, small displays and low memory.Secure Service Access from Diverse ClientsTrusted active proxies will convert sensible data to reduce the value before displaying on mobile devices. Therefore the chance that some unauthorized person can view sensible data is reduced. Another method is to use “one time passwords”. Further, they provide an adaptation of the SSL standard for devices with low computation capabilities.Multiple Device FusionProxies are able to combine devices for content and security adaption e.g. a user can input sensitive personal data with a PDA but view the content on a machine with bigger screen and better GUI.2.3.4 PATHSThe NINJA project allows the dynamic composition of horizontal services into a path which is very flexible and supports enhanced functionality. A path is a flow of data through multiple services including transformation of the data from one service to the other into the needed data formats. Therefore services have defined interfaces for connecting each other on a path.The path creation process is dynamic and a path can change during its lifetime. Different paths can be compared by their costs e.g. quality of service or resource consumption or data flow speed and the best suiting path will be chosen. If e.g. a network link becomes overloaded the data flow can be redirected and therefore the path will change at lifetime.A necessary step of forming a path is being able to locate services to place in that path. The NINJA architecture includes a SDS (service discovery service) that allows both human users and programs to locate appropriate services across the wide area based on service attribute queries. All services publish descriptions of themselves to SDS instances running in their local base. Summary information about that known services is exchanged through the SDS infrastructure. Searches propagate through the SDS hierarchy until matching information is found.2.4 Programming Model and Design PatternsInternet services must be able to handle high throughput maybe thousands of requests per second and more. A service must remain robust even under that workload. If the load exceeds the service capacity the service must handle this situation too. To achieve this goal the NINJA project offers certain design patterns for service authors and a particular abstraction on services which mean that services can be split up into stages. This means that these stages can be isolated from each other and perhaps be physically separated across address space or physical machine boundaries.There are four offered design patterns:WrapThe wrap pattern places a queue in front of a stage and assigns some threads to the stage to process tasks that arrive on the queue. Excess work that cannot be taken by the stage immediately will be buffered in the queue and processed later. The stage can decide then what tasks to execute and therefore can give higher priority to some tasks and even drop other tasks in case of overload or no admission for the task to execute.PipelineThe pipeline pattern takes a wrapped stage, and splits it into two pipelined, wrapped stages. This decouples a stage and allows parallelism across processors or clusternodes. This permits further optimization such as having a single thread repeatedly execute the same code and increasing instruction locality.CombineThe combine pattern is the inverse to the pipeline pattern. It fuses the code of two former splitted stages into one wrapped stage with one queue. The combine pattern permits resource sharing between these previously independent stages.ReplicateThe replicate pattern duplicates a wrapped stage on a number of independent processors or cluster nodes. This is used to eliminate bottlenecks and increasing the throughput of a stage.NINJA offers these design patterns through a programming library to ease the process of software design and execution. With this library a service can be conditioned to best performance and stability and fault tolerance.NINJA was designed using JAVA JDK version 1.1. Optimizing compilers e.g. IBM JIT and OpenJIT were used for a higher performance. Because of JAVA the NINJA project is almost platform independent. Many programming issues and sources for bugs and problems are eliminated by language properties of the JAVA programming language.2.5 vSpace Execution PlatformvSpace is an execution environment for scalable Internet services which operates on a cluster of workstations. vSpace services are constructed using the before mentioned design patterns. As services are started their code and composition will be freezed so if code will be changed the old services will continue to run and only the new started services will include the changes in the code. The vSpace execution environment can replicate services on demand to cope with higher system load and can destroy such called clones, if they are not needed anymore. vSpace therefore is a highly scalable execution environment. It also provides mechanism for failure detection and clone restarting.2.6 Distributed Data StructuresThe Distributed Data Structures (DDS) are self managing storage layers designed to run on a cluster of workstations. It has high throughput, high concurrency, availability, incrementally scalability and strict consistency of its data.Service authors see the DDS as a conventional data structure and don’t have to deal with all these just mentioned functionality. The DDS platform hides all those features from the service author and simplifies the service design process.2.7 Service Discovery Service (SDS)The Service Discovery Service (SDS) provides two functions. First it provides a mechanism by which services can announce their presence to the infrastructure and second it provides a mechanism by which both human users and programs can locate these announced services across the network. The SDS is a scalable, fault-tolerant, and secure information repository.At the SDS three different situations have to be managed:Clients want to discover servicesSDS servers collecting/providing service informationServices themselvesThe SDS saves queries and service information as XML data. Using XML helps in validation of service descriptions and it is flexible to expand existing service descriptions. Tests have shown that a SDS server based on a PentiumII could handle about 500 clients sending queries of one query per minute per client2.8 NINJA Example ServicesThe NINJA project team developed a few practical examples using the NINJA framework. These examples are:The Ninja JukeboxThe Jukebox is a distributed repository of digital music with a dynamic music directory, available at the campus network. It can stream mp3 to mobile clientsActive Proxy FrameworkThis application was designed for accessing secure data through clients e.g. stock data.Ninja MailThis project wants to build a scalable and feature rich e-mail service on top of NINJA. Ninja Mail is a general e-mail infrastructure which other applications or services could use.3 OSGI FrameworkThe OSGI Framework is developed and supervised by the OSGI Alliance, San Ramon, California, USA. Their website can be found at These companies are members of the OSGi Alliance: 4DHomeNet, Alpine Electronics Europe, AMI-C, Aplix, Atinav, Belgacom, BMW Group, Cablevision Systems, Deutsche Telekom, Echelon, Electricite de France (EDF), Esmertec, Espial Group, France Telecom, Fraunhofer Institut, Gatespace Telematics, Gemplus, IBM, Insignia Solutions, Institute for Infocomm Research, KDDI R&D Laboratories, Mitsubishi Electric, Motorola, Nec, Nokia, NTT, Oracle, Panasonic Technologies, Philips Consumer Electronics, ProSyst Software, Robert Bosch, Samsung Electronics, Sharp, Siemens, Sun Microsystems, Telcordia Technologies, Telefonica I+D, TeliaSonera, Texas Instruments, Toshiba.3.1 OSGI OverviewThe OSGi specifications define a standardized, component oriented, computing environment for networked services. Adding an OSGi Service Platform to a networked device (embedded as well as servers), adds the capability to manage the life cycle of the software components in the device from anywhere in the network. Software components can be installed, updated, or removed on the fly without ever having to disrupt the operation of the device.Software components are libraries or applications that can dynamically discover and use other components. Software components can be bought off the shelf or developed in house. The OSGi Alliance has developed many standard component interfaces that are available from common functions like HTTP servers, configuration, logging, security, user administration, XML, and many more. Plug compatible implementations of these components can be obtained from different vendors with different optimizations.initially targeted at residential Internet gateways with Home Automation applicationsNokia and Motorola are driving an OSGi based standard for the next generation smart phones.The vehicle industry adopted the OSGi specifications by making it an intrinsic part of the AMI-C specifications, which is supported by many carmanufacturers.Also, the OSGi Service Platform has become a standard part of the BMW high-end telematics platform3.2 OSGI IntroductionThe OSGi specifications are so widely applicable because it is a small layer that allows multiple, Java based, components to efficiently cooperate in a single Java Virtual Machine (JVM).It provides an extensive security model so that components can run in a shielded environment.However, with the proper permissions, components can reuse and cooperate.The presence of OSGi based middleware in many different industries is creating a large software market for OSGi software components. The definition of the OSGi Service Platform enables components that can run on a variety of devices, from very small to very big.Adoption of the OSGi specifications can therefore reduce software development and maintenance costs as well provide new business opportunities.proprietary service platform spans:Digital mobile phonesAutomotiveTelematicsEmbedded appliancesResidential gatewaysIndustrial computersDesktop PCsHigh-end servers, including mainframesOSGI framework provides a standard, non-proprietary, software component framework for manufacturers, service providers, and developers. The fact that the OSGi specifications are an open standard enables a fair playing field for all participants.It provides a powerful model for co-existence of different components/applications in a single JVM. Running multiple applications in the same JVM minimizes the memory footprint, increases performance, and provides near zero-cost inter-application communication.The framework has a flexible deployment Application Programming Interface (API) that controls the life cycle of applications. Applications are installed with a standardized deployment format and can then be started, stopped, updated, and uninstalled without requiring the JVM to be restarted. This is a must for continuous computing.OSGI is a secure environment that executes applications in a sandbox so that these applications cannot harm the environment, nor interfere with other resident applications. This model allows less trusted applications to run inside the JVM without compromising the overall integrity of the system.It has a cooperative model where applications can dynamically discover and use services provided by other applications running inside the same OSGi Service Platform. This cooperative service model allows OSGi applications to be much smaller than applications for other Java application servers. This is a key benefit for mobile devices.The framework has a flexible remote management architecture that allows platform operators (the organization that manages the platform) and enterprises to manage thousands or even millions of Service Platforms from a single management domain. The OSGi Service Platform architecture allows the operator to control a platform in fine detail by using a model where the operators can ensure that their required policies are implemented, however secure or flexible those policies are desired to be.OSGI framework consists of a number of standardized, optional services: Logging, Configuration, Http, XML, Wiring, IO, Device Discovery and driver loading, User authentication and authorization, Preferences, and UPnP. Additionally, it provides a number of standardized utilities.3.3 Key Features of the Service PlatformSoftware Component Management• A packaging format for the applications. The OSGi specifications provide the Bundle format. Bundles are applications packaged in astandard Java Archive (JAR) file, which format is fully compatible withthe ubiquitous ZIP files.•Install a bundle. The bundle must be prepared and the diversecomponents installed in the OSGi Framework, ready to be executed.•Start/Stop a bundle. Installed bundles can be started and stopped in an OSGi Framework. Starting a bundle makes certain resources available,stopping a bundle cleans up. In an OSGi Service Platform, allapplications are started in the same JVM, thereby saving memory,resources, and CPU cycles.•Update a bundle•Uninstall a bundleSecure Execution Environment•VM security mechanism•Safety feature of JAVA language itself•JAVA 2 code based security•OSGi Framework that strictly separates bundles from each other.Cooperation Between Applications•Sharing of libraries (save memory), find and publish functions Commercial Off the Shelf Components•Since standard environment a lot of packages available Simplified Deployment•simplify the process of deployment.Multi Vendor Interoperability•gives the platform operator as well as the manufacturer a choice. The operators can buy components from different vendors and run them onthe same service platforms. The resulting competition will increasequality and lower prices.Dynamic Nature•standardized hot update technology for small embedded devices, desktops, and servers enabling these devices to run continuously, evenwhen their software is configured or updated.3.4 OSGI ArchitectureThe OSGI Architecture consists of several layers. At the bottom is the OS Hardware and the running Java Virtual Machine. On top of that are the following OSGI layers:Services Service Registry Live Cycle Class Loading SecurityApplications/BundlesJVMOS HardwareThese layers will be described in the following.3.5 OSGI Services3.5.1 Framework ServicesPermission Admin service•The permissions of current or future bundles can be manipulated through this service.Package Admin service•Bundles share packages with classes and resources. This service provides information about the actual package sharing state of thesystemURL Handlers service•The URL Handlers service enables bundles to dynamically contribute new scheme or content handlers to the URL class.Start Level service•The Start Level service sets the current start level, assigns a bundle toa start level, and interrogates the current settings.3.5.2 System ServicesSystem Services provide horizontal functions that are necessary in many systems.The Log ServiceConfiguration Admin serviceDevice Access service•Device Access is the OSGi mechanism to match a driver to a new device and automatically download a bundle implementing this driver.This is used for Plug and Play scenarios.User Admin serviceIO Connector servicePreferences Service3.5.3 Protocol ServicesOSGi Alliance has defined some services that map an external protocol to an OSGi service.Http Service•The dynamic update facility of the OSGi Service Platform makes the Http Service a very attractive web server that can be updated with newservlets, remotely if necessary, without requiring a restart.• UPnP•Universal Plug and Play (UPnP) is an emerging standard for consumer electronics. The OSGi UPnP service maps devices on a UPnP networkto the Service Registry. Alternatively, it can map OSGi services to theUPnP network.3.5.4 Miscellaneous ServicesWire Admin connects different services together as defined in a configuration file. The Wire Admin service uses the concept of a Consumer and Producerservice that interchange objects over a wire.XML Parser – The XML Parser service allows a bundle to locate a parser with desired properties and compatibility with JAXP.4 ConclusionProblems of Mobile Computing and of providing mobile services are e.g. the different clients with small displays, weak computing power, low memory, discontinued connection to the network and services, geographically moving devices, cheap devices that can be lost or broken and all data has to be reconstructed through the services. Another big issue are security concerns. Both frameworks try to deal with these problems in a different and unique way.The NINJA framework was developed at a university and relies on a cluster of workstations for the service bases. It can deal with a wide variety of clients and is highly scalable because of its architecture and highly portable because it was developed in JAVA. The source code is available for free and everybody can develop extensions to the NINJA framework. The University of Berkeley has developed some sample applications to show how their framework works and for performance tests.I think the NINJA framework could be very successful if more people would use the framework. Unfortunately it is not being used by a lot of people and therefore has a small community. The NINJA framework will not be developed and improved any further at the moment and I think this is very sad because of the interesting and innovative way they solved the problems.The OSGI framework is more or less an industry standard with a lot of companies as members of the OSGI Alliance. So a lot of companies and people use the OSGI framework and therefore there are a lot of practical implementations using the framework. It is widely used e.g. in modern smart phones (Motorola), in vehicles (BMW), in industrial automation and even in software architecture (Eclipse). Because of so many companies being certified by the OSGI and because there is much money put into development of the platform it will have a better future than the NINJA framework.This is sad and I think the NINJA platform should have a chance, too. But because of the lack of money the NINJA platform will not be developed any further. I think the idea of the NINJA framework and the architecture and design of the NINJA platform are very good. So if a big company would push the NINJA platform with money and people it could have a future!。