Netflix Phenomenon and Mobility

Three weeks ago, I attended a two day workshop called the mGov Strategy.  This workshop's purpose is to provide input to OMB's mobility strategy for the US Federal Government.  Steve VanRoekel, OMB CIO, sponsored this workshop.  US civil servants from various lines of government joined the workshop. OMB split the working group into five sub-groups.  The sub-groups were:

1. Acquisition - How can US government address acquisition of mobile technologies and services? Can the US government streamline the acquisition process.
2. Security - How to safeguard government information and technologies from hackers.
3. Privacy - How to protect mobile user information from inappropriate use especially when they interface with US government mobile sites and apps.
4. Citizen apps - How to develop a mobile presence to engage US government's biggest customer US citizen.
5. Infrastructure - How can US government address the evolution of mobile technologies and associated technologies like cloud computing, social computing and others.

As the member of the security sub-group, we discussed several policy and technical approaches.  The thing that caught my eye and basically sums up any future technical advancement is the ability to do use any application from anywhere and anytime.  I call this the "Netflix phenomenon"

Even though Reed Hastings, Netflix CEO, won't win the CEO of the year award, I still give him credit in taking the movie watching experience from a cinema theater to any possible device which is accredited by Netflix.  I admit that Google introduced this feature with YouTube however Netflix took it to a new level.  I can now start a movie via  my laptop, pause it and then resume it on my iPad.  I like this DirecTV commerical which captures what I am talking about.


To develop this type of an IT service, enterprises need to invest in the following technologies and architectures like:

• Cloud computing - IT departments need to centralize their business applications and act as cloud brokers to outsource some of their applications to third party clouds like Amazon EC2, Google Cloud, Rackspace and others.  I believe unless OMB makes significant investments in IT infrastructure, agencies will have to act as cloud brokers. It's a cost effective mechanism.
• Smarter Pipes - Where is Mario when you need him?  With all of the data and information streaming back and forth between clouds, user devices, government needs to influence how IT networks should evolve.  Since mobile users are constantly starving for the fastest network, vendors have to realize that simply scaling up the networks is not a sustainable model.  Vendors and research institutes need to look at how data should traverse the network and optimize it.  A good example is that vendors need to develop information caching mechanisms at the network level.  
• Smarter security - One of the best phrases used by the mobile users in the government space is, "brick". Users can call and email on a brick but nothing else.  Security personnel should realize that clamping everything defeats the purpose.  IT risk management should be a key in developing a smarter security posture. Single sign on is key as well. No one wants to sign on with multiple usernames and passwords to do their work.
• Usability - One of the best parts of using Netflix is how intuitive the user interface is.  Ease of use is a key phrase to describe Netflix's user interface.  We need to identify and prioritize what functionality is needed or desired on a mobile app.
• Flexibility - Use sound architecture principles like loose coupling, simple interfaces and architectures.  Simpler is better.  
• Standards based architecture -  Eventhough there is an over abundance of  standards especially XML (frankly I am sick of how folks are misusing it), we still need to emphasize it and design appropriately.  Having a 50MB XML payload in SOA enabled information exchange is NOT smart architecture.  I am not going to expound on the 50MB XML example since it is aggravating.

Even though Reed Hastings didn't make alot friends with Qwikster or jacking up the monthly Netflix fees,  he did build a pretty cool service called Netflix streaming.  As I write this blog, my youngest son is watching Power Rangers in Space via the WII and my oldest two are watching a Dreamworks movie via the PS3.  My third one is having fun the old fashioned way. She is attending a birthday party. Thanks Reed and now bring down the monthly fees.

XML Driven Software Architecture

As technologies are evolving from a procedural to object oriented and architectural principals as loose coupled and flexible software is ideal, developers are constantly thinking or taught to think that abstraction is good and brittle code is bad. This idea has also evolved in Information Technology (IT) enterprises where the notion of a service has caught. Since IT enterprises involve more than hard-core java drinking code junkies, services have evolved into business services where services abstract business processes from their supporting IT systems. As Business Processes are changing with respec... WAIT!!! This is a Technie blog entry so lets stay in the realm of software, hardware and whatever makes a java drinking, red-bull drinking, dry cereal snacking technie happy.

As software architects and developers are praising the likes of Gavin King, who developed the Object Relational Mapping (ORM) technology Hibernate, since they developed technology which abstracts developers from knowing how the data is stored in a relational database. ORM technologies allow programmers think in objects rather than normalized relational tables with primary and foreign keys. The wide argument is that programmers won't get bogged down by writing sql queries but they can simply think of their code as objects. The problem with this notion is that sometimes ORM queries can be poorly written.

As I was pondering this issue today, I realized that ORM still couples the software code like Java to the structure of the database. From my experience, when new tables or fields were created in the database, the ORM code has to be modified and this in turn may affect the actual business logic code. This could get painful when new requirements or the Database Administrator (DBA) has provided a new design to optimize the database.

One way to get around this issue to design the business datamodel as a XML Schemas (XSD). After working with XSDs, I believe they do a great job of capturing the object model. After creating the XSD, generate XMLBeans or JAXB classes and code to these classes. The JDBC code or ORM code can be then communicate to the business logic tier via the XML binding classes. This way the XML binding classes provide a layer of abstraction between the data tier and the business logic tier. The problem with this approach is that XSD can be hairy however if you understand XML and XML technologies than this may be the way to go. The other issue is that there could be issues in the generate classes or how you created the XSD. Once again if you are comfortable with XML then may be a way to go since I think it could be better than ORM.

Please let me know if you disagree with me.

XML Specification and Its Implementations

Since XML is a specification and not a technology, technologies which work with XML are implemented different. This is most evident in the new technologies like Semantic Web and Services Oriented Architecture (SOA) which extensively use XML. For example:

Semantic Web
In November 2007, I attended an ontology conference in Columbia, MD. During the conference, Steven Robertshaw stated that there is no standard ontology editor since each editor renders, validates a OWL document differently. OWL is a W3C standard for ontologies. I agree with Robertshaw's statement because the OWL document which I created from Protege did not validate on Altova's SemanticWorks OWL editor. When I fixed the OWL document for the SemanticWorks tool then it wouldn't work on Protege.

Web Services
Altova's XMLSpy generated WSDL does not validate well on Cape Clear's WSDL editor called SOAEditor. WSDL is a W3C approved XML specification for web service contracts. Sometimes certain editors look for attributes in a different place. I have used Mindreef's SoapScope to test a WSDL which is approved by all WSDL editors.

XML Schemas
While I was doing in depth analysis on Global Justice XML Data Model (GJXDM) for Department of Homeland Security, I created XML Schemas using XMLSpy . This caused a major issue in XMLSpy editor. The graphical interface stated the the xsd was valid however the xsd was invalid in the text view. After some research, I determined that XMLSpy used multiple xsd validators and that each XMLSpy view used a different xsd validator. This is still a problem. Look at this article.

Going back to my last blog entry, XML based modeling technologies have too many quirks and this is because how the technologies have implemented various XML specifications. This type of confusion is NOT needed by a data modeler. Before I start any modeling exercise, I would say, "Where is my Protege? "

Resolution for 2008: Lets cut down on XML abuse

When XML started off it, it was envisioned to be a super set of HTML where custom tags could get created and content in custom document strutures can be circulated in the World Wide Web. I, however, believe that XML technology is being misused. It is popping up in Semantic Web, SOA, and in pure data modeling efforts. It is true that XML is human AND machine readable however computing processors are paying a heavy price parsing and ingesting the content in complex XML documents. I recently attended an Ontology conference in Maryland and the biggest theme from the conference is that there isn't enough machine power to process all of the complex business logic to make content inferences. Even though XML is human and machine readable, it is extremely verbose and it is not practically to have machine process all of the tags and get the data from a XML document. The next question to ask is: "When should you use XML?" The answer is that when the XML documents are not overly complicated which might include nested tags, and a large amount of data. If the data is quite big then other binary content should be passed around in COM objects or Enterprise Java Beans (EJB)s. Here are some arguments regarding XML

1. XML is interoperable - True XML is interoperable for cross platform communication. However please don't send massive XML documents which can bough the system down. XML may be interoperable but it is not performance friendly.
2. XML is a good format for web service communication - Web Services (SOAP and REST) largely deal with XML based content however Web Services are NOT reusable if the XML format is propriatory between the Service Provider and its clients. Please avoid sending large amounts of data over the wire. SOAP with attachments is a good alternative for sending large chunks of data.
3. XML is human readable - This is true however if a developer decides to create a XML format which he only understands then the XML is unreadable and it needs to be released. For example, a developer decides a create a XML document which looks like this: Person. This XML document is a complete waste since not every human understands it. However if tag names are defined properly and their definitions captured then the XML is human reable.
4. XML is the future in Semantic Web- Recently I came across OWL-S editor software and I tested it out. The software generates an OWL and RDF documents however these XML documents are verbose and they cannot be processed. Instead of a XML based technology, the Semantic Web should look at other options rather than just XML.

In conclusion, XML is a great technology however people tend to misuse it and then tag it as a wasted technology. XML is not going to go away so lets use it properly and take care of it.

Are RESTful web services ideal in a SOA?

Representational State Transfer (RESTful) web services are the latest buzz in the web services domain. What are RESTful web services? RESTful web services consists of HTTP clients which send requests as request parameters in the URL and the response is a XML document which can be viewed in a browser. This is ideal service for technologies like Asychronous Javascript and XML (AJAX) which take XML responses and apply XML transforms (XSLT) to generate a visually pleasing display of content. AJAX's primary mode of transport is through the HTTP browser and it's mode of transport is HTTP or HTTPS. Currently other developers and vendors like Yahoo!, Google and Microsoft have harnessed the AJAX technologies, RESTful services and RSS feeds to generate data aggregators which allow users to find, aggregator, and filter from various data sources how ever they are merely point-to-point services. AJAX working with RESTful web services is the ideal model for the federated query where one request is federated across multiple data sources and then the responses from each data source is aggregated to show the "full" picture.

Yes RESTful web services are not that bulky since they have don't have the SOAP wrapper however let us not get caught up in RESTful services are ideal in a SOA enterprise. The problem with REST is that it is designed for point-to-point services with minimal reuse. This is the same type of problem where SOAP based services exposed method calls which required specific datatypes in their Web Services Description Language (WSDL). When it is a point-to-point services, alot of work is spent in formatting, transformation and processing information to fit the specific datatypes in the non-reusable methods. To make a SOAP based web service more SOA friendly, it is recommended that the services take XML documents as the parameters in the web services method calls. This way one XML document can be passed around where the contents of the XML document might be altered by various method calls. It is not practical for RESTful web services to pass a reference to a XML document as a request parameter.

RESTful web services are great with AJAX however I wouldn't recommend using RESTful web services for services which are highly popular, reliable and have a potential to be reused alot.