Assessing IT Trends via Google Trends

After spending more than eleven years in the Information Technology, it is amazing to see how technologies evolve and trends are created and destroyed. I did a workshop on Thursday March 17,2011 with my colleague, Mr. Giora Hadar, on social media at FAA's IT\ISS conference where I discussed the evolution of social media and how FAA is using it. While preparing for the presentation, I realized how US government information policy is influenced by technology trends. Anyway here are some trends as viewed on Google Trends:

• Query: "cloud computing"
  
• Definition: "Cloud computing refers to the provision of computational resources on demand via a network." - Wikipedia
• Google Trends:

  

• Query: "Service oriented architecture"
• Definition: "Service-oriented architecture (SOA) is a flexible set of design principles used during the phases of systems development and integration in computing." - Wikipedia
• Google Trends:

  

• Query: "social media"
• Definition: "Social media are media for social interaction, using highly accessible and scalable communication techniques. Social media is the use of web-based and mobile technologies to turn communication into interactive dialogue." - Wikipedia
• Google Trends:

  

• Query: "bpms"
• Definition: "Some define the BPM System or Suite (BPMS) as "the whole of BPM." Others will relate the important concept of information moving between enterprise software packages and immediately think of Service Oriented Architecture (SOA)." - Wikipedia
• Google Trends:
  

  

• Query: "knowledge management"
• Definition: "Knowledge Management (KM) comprises a range of strategies and practices used in an organization to identify, create, represent, distribute, and enable adoption of insights and experiences." - Wikipedia
• Google Trends:

  

• Query: "enterprise architecture"
• Definition: "An enterprise architecture (EA) is a rigorous description of the structure of an enterprise, which comprises enterprise components (business entities), the externally visible properties of those components, and the relationships (e.g. the behavior) between them. EA describes the terminology, the composition of enterprise components, and their relationships with the external environment, and the guiding principles for the requirement (analysis), design, and evolution of an enterprise.^[1][2][3]" - Wikipedia
  
• Google Trends:

  
  

As you can see, "cloud computing" and "social media" are the hottest IT trends while "service oriented architecture","knowledge management" and "enterprise architecture", are declining. BPMS is also on the rise. What this tells me that CIO will try to align their strategies to the latest trends. These trends correlate to the Gartner Hype Cycle. I wonder if folks are doing any analytical strategic work using a tool like Google Trends.

SOA will die!

I think Services Oriented Architecture (SOA) will die with the whole notion of Cloud computing. If you were to follow to the history of Google APIs (Application Programming Interface), you will notice that Google had quite a few XML based Simple Object Access Protocol (SOAP) web services in the 2003 - 2004 time frame. If you were to surf the Google Code base, you will notice that there aren't alot of web services enabled APIs. What does this mean?

I think with phenomenon like Cloud computing, there will be a decrease in system to system level integration. How many of us can say that Enterprise Java Beans (EJBs) will make a come back? Maybe one or two handfuls around. The reality is that IT is evolving so fast that complexity of the accessible API is more simplified. Who would ever have thought that JavaScript is the favored scripting language to access APIs? It does make sense though. JavaScript is here to stay as long as the web browser (workstation or mobile). In this ever changing evolving IT world, services will be designed and developed using JavaScript. I am waiting for a JavaScript specifically for system integration. I like the JavaScript libraries that were built for Facebook and Google. Goodbye SOA and welcome to JSOA aka JavaScript Oriented Architecture in the cloud. I believe cloud will also cause the fall of the relational database management system (RDBMS) and rise to the Reduced HashMap. I will expound on this evolution in the data tier in the near future.

The image is taken from David Chou's blog entry, "Cloud Computing and the Microsoft Platform" at http://ht.ly/3ACCE

Make "TRON" interoperable

Yesterday I went to the movies with my son Seth. We had a nice Mediterranean dinner. After which, we got Edy's Mint Dibs ice cream and medium Barq's root beer and then we settled in our chairs and watched the Disney movie Tron:Legacy. It brought memories of me when I was 13 year old boy watching the original Tron. I thought it was the coolest movie every. I now have to say this movie is actually not that bad. My son liked the movie too.

Anyway the point of this blog entry is not to expound on the entertainment value of the movie but rather philosophical flaws of the movie. The human characters interact with one system and its programs. The "Tron" world is a universe of one system. This is so 1990s. Is this movie promoting stove-piped systems? I don't think so but it does show you the inherit flaws of a stove-piped systems. Here are the flaws:

• One world concept - A stove-piped system is a world on its own. The language, the business logic, and hardware and software are designed for that system.
• Limited scalability - Any optimization and scaling is done within the constraints of the system. My world is the center of the universe. Oh wait, my world (aka my stove-piped system) is the universe.
• Failure to Adapt - In the movie, there is no sun. Hence we can infer that the "Tron" system doesn't have a program which acts like a sun. Now lets say there is a "Sun" program introduced into the "Tron" system. Will other programs recognize there is infact a "Sun" in the "Tron" system? Probably not because that logic has not be programmed in.
• Reusable components - Imagine if the "Tron" system had reusable programs which can be used in several other systems. If so how can these programs be ported into other systems. The program "Tron" is imported in the "Tron" system however this program had re-programmed to interact in the "Tron" system.

Anyway if there is a Disney producer or a script writer reading this entry then please consider the following ideas for the next Tron movie:

• Introduce programs called Interfaces which allow the Tron world to interface with other worlds like the "blue world" (IBM) and the "red world" (Oracle).
• A cyber war across the network (kinda like an alien war)
  
• Virus software in the Tron world
• Tron on Facebook or MySpace - just kidding

Anyway the movie was great but the movie also shows the flaws in having "stove piped" systems.
Here is the trailer

What is a Service?

To understand Services Oriented Architecture (SOA), we must understand what a Service is. In the world of buzz word confusion, the notion of SOA is misunderstood which in-turn hurts the Information Technology (IT) component a business which in-turn hurts the business itself.

Therefore the components of any Service (business service, web service, project management service, aviation service or security service, etc., and etc.) should be identified and modeled. All Services contain the following:

• Inputs
• Outputs
• An Interface which facilitate the inputs and outputs
• Cost associate in conceiving, designing, implementing and maintaining this service.

Here are also some characteristics of any Service?

• How is the Service invoked? – Service Invocation
• How is the Service delivered? – Service Delivery
• How good is the Service? - Quality of Service
• How is the Service supported – Service Support

Even though all Services have the common components and characteristics, it is important to understand that Services are realized within a context and some characteristics or attributes become mandatory. For example within the context of technical solutions architecture, Services can be realized as Web Services, Database Services, or any other Technical Service. Technical Services usually have well defined Inputs, Outputs, Service Interfaces and the Service Cost which is usually associated with the licensing of the technical infrastructure or the cost of building the technical capability. For example, the term “network protocol” is big within this context since it describes the underlying architecture of the Service Delivery mechanism.

If the context is a Business Automation process, then Business Automation Service usually involve the application of Technical Services to address how a certain business process or a sub-process is automated. Since the application of Technical Services and Technical Services may sound similar, they are uniquely different. The lack of clear delineation of what a Business Automation Service is and what a Technical Service is one of the common underlying problems within SOA. There are also Services within the context of a Business. For instance, Oracle Corporation sells Support Services for its products. The Support Services usually include Help Desk personnel who answer and guide Oracle Support Services’ Consumers on how to resolve issues with the Oracle product. Oracle and others have tired to automate this component in the Support Services model however the “human touch” and human interaction is highly desired in the Support Services Consumers’ community. The Support Services Consumer invokes the service when he or she is having issues with the Oracle product, which has Support Services attached to it. The Service Consumer uses the phone or website interaction as his or her interface with the Oracle Support Services. The Service Consumer doesn’t need to know who the other person is, how is he getting paid, what are the Support Services’ processes are, etc., etc. Now the question I often ponder about is:
“Can a business be truly SOA with respect to various contexts and not just IT?”

I believe it can be done however will the Return On Investment worth this change.

Can SOA be achieved in an enterprise

For the last year or so, I have been working for the Federal Aviation Administration (FAA) as a Services Oriented Architecture (SOA) Subject Matter Expert (SME). It has been a challenge to see how various folks view SOA the panacea for IT related systems or enterprises. Each stakeholder within an organization has different definition with respect to SOA. So far I have noticed the following at the FAA, we have:

1. Data Services - Services that carry data
2. Business Services - Services that meet a business need
3. ITIL Services - Services which meet Help Desk and Operations centric services
4. Business Process Services - Services which emulate business processes and they use standards like BPEL and BPMN
   
5. Web Services - Information is exchanged over the HTTP protocol in XML or as attachments which are in SOAP container
   
6. Java Services - Java enabled services
7. RESTful Services - Information is exchanged over the HTTP protocol in XML but the information is not in a SOAP container.
8. Orchesterated Services - Services which are made of services
9. etc., etc

As you can see, Services can be confusing. To add to the confusion, we have other tools and methodologies which make SOA a zoo. We also have people who view SOA serving different purposes. In the end, they are all true since they are identifying components which can be or may be true in an enterprise. Here is my take on SOA. SOA should be viewed from a Enterprise Architecture (EA). EA identifies (if it is done right) the "As-Is" state of the IT enterprise and proposes the "To-Be" state of the IT enterprise.

EA should also have a strategy on how to achieve "To-Be" state in a timeframe. This is usually articulated in various EA artifacts like Roadmaps, line of sight documents, strategy documents and an EA repository. Other components that need to implemented to give a true EA picture is to have an enforced System Development Lifecycle (SDLC), Change Control Board (CCB), and Configuration/Change Management (CM). The other thing is that to have a working EA, all of the stakeholders must buy into it. I am a big fan of The Open Group Architecture Framework (TOGAF).

Back to SOA now, if EA recommends that SOA is one of the possible approaches to meet "To-Be" state then it needs to bought by all stakeholders in the enterprise. The following steps need to be done when an enterprise wants to use SOA to achieve its goals:

1. Define a Service Data Dictionary - This will reduce the confusions when talking about services.
2. See if the EA strategies identify any systems which need to be consolidated or broken-up.
3. If so then use SOA in system(s) migration
4. If all new systems are built in a SOA fashion then address SOA governance
   
5. Address SOA governance by designing and implementing processes and groups which deal with SOA governance.
6. All new SOA services should be projectized
7. All new SOA services should have a Mission and Vision statement
8. and on, on

In the end, will SOA ever be truly achieved in the FAA. The answer is "No"! However pursuing SOA will help FAA identify gaps in its enterprise with respect to its EA since it forces various stakeholders to communicate together and address how some of the internal processes should be achieved. This is a big key for any successful IT enterprise. Adopting SOA will not provide a successful EA without having a working SDLC and appropriate mechanisms which provide accountability.

That was Suite! Not really

This morning I was asked to go to an all morning presentation by Oracle. The presentation explained Oracle's vision with its products and its newly acquired company's, BEA, products. The core presentation was given by Thomas Kurian Senior Vice President for Oracle's Fusion Middleware. The presentation was insightful and it showed that Oracle has a defined and definite vision. I, however, being a skeptic will accept Oracle's vision when I see results. Other than enjoying Oracle's flagship product which is their relational database product, I have not been impressed with Oracle products. That being said, I do believe that Oracle is going to keep the Weblogic Application Server as well as other quality products. It is going to be interesting how other vendors will compete with Oracle in the Application and Middleware space.

Like Oracle, IBM has been buying strategic companies like Oracle but they NOT are buying monster size companies like BEA or PeopleSoft. IBM products work well however they are very expensive and their support is expensive as well. Oracle is following the Sun and RedHat approach which is to follow the Open Source paradigm. Thomas Kurian stated that Oracle is going to streamline the Weblogic IDE,which is based on Eclipse framework, make it open source. It will be interesting where Oracle Fusion Middleware will be in the next two to five years. During the presentation, Thomas Kurian talked about the Weblogic Suite, SOA Suite, BPM Suite and every other Suite possible. So in closing I say, "That's suite!" Actually I say this Larry Ellison, show me good software and I will say, "Suite software". If the software is not good then I will call the Oracle Fusion Middleware, law-suit software and not suite software.

Zapped

Last week, I was in a ZapThink Service Oriented Architecture (SOA) Bootcamp. It was a descent class since they derive their material from actual SOA implementations in large corporations. They presented high level principals and "gotchas" but I felt it lacked something but I cannot tell what. I thought the ITIL v3 Foundations course was more intense and it actually presented some excellent concepts. I highly recommend taking ITIL v3 Foundations course and the ZapThink bootcamp does a good job of re-enforcing overall same concepts which are:

• How does IT solve a business problem? This is key since vendors and developers tend to buy the tool or solution and then retrofit it to a business problem. I have been involved in couple of projects which spelled D-O-O-M-E-D after they were cancelled.
• Capture Business Processes - This is an other key since requirements are derived by analysts which inturn are passed down to the architects and engineers. Architects and Engineers design solutions to the requirements. If they had an insight into the business processes, the solutions could be better and cheaper.
• Business Agility - Flexible IT systems provide businesses more flexibility to be agile. Services support this paradigm however defined Configuration Management processes are key as well.
• Architecture - Enteprise, Data, Process, and Technical Architectures are key since they provide a defined approach in designing a system or an enterprise. No more Ad-hoc development projects. As a developer and an as an architect, this is key since developers love to know their constraints. We, as developers, have been burnt by poorly written code or no documentation.

Overall good IT architecture with SOA or no SOA, involves alot of common sense, a business understanding and knowing your constraints. Ideas are great but we all have to be realistic.

IT Architecture = Communications

For the last six months or so, I have been working for the US Government as a Services Oriented Architecture (SOA) Technical Lead. Eventhough that is my functional title, I act more as a Information Technology (IT) architect. It is my first job where I am an IT architect. At my previous job I was a part-time XML data architect where I did alot of data modeling, data analysis and then defend why I modeled certain data structures a certain way. When I was not a XML data architect, I was a JEE developer. Eventhough developers aspire to be an architect because they can get to design the systems and leave the grunt work to other developers, this may not be true when the developer becomes an architect.

Couple of summers ago, I took a class to become a Java Enterprise Architect and it is there I learnt that an architect doesn't have a set role. As a developer or a project manager, your roles are pretty much defined but an architect's role is ambigous. In the class, I was taught that one of primary responsibilities of an architect is to see his system or project succeed. In some projects, he can be coding and developing reference architectures and in other projects, he is basically working with the project manager in refining requirements. After being an IT architect for the last six months, I have to say that the instructor was absolutely correct. An architect's role is ambigious to a project but every project must have an architect. An architect is a rare breed because:

1. he understands what is being built and why it is being built
   
2. he can look at code and have code level discussions about the software code.
3. he understands the business problem and why the system is a value to the customer
4. he can work with business analysts in refining requirements or make them more abstract which might give the development team more flexibility
5. he can sit with the project manager and hash out a realistic Work Breakdown Structure (WBS)
6. he can recommend modifications to the project scope if the project is running behind schedule
7. he doesn't need to be the best programmer in the team
8. he doesn't need to design the system but rather review them
9. he doesn't need to write a test plan but he can evaluate it
10. he doesn't need to be a guy. Women can be architects too.

And the biggest trait an architect should have is that he or she should be able to

COMMUNICATE

with various stakeholders and he or she should be able to discuss the system in technical, non-technical, and business terms. Architects should be willing to exchange ideas with other architects and they should humble enough to accepts mistakes if they made them.

The role of an IT architecture will expand as more businesses go into collaborative modes of information exchange since they need to be aware of standards and they should understand the newer technologies. So for all aspiring programmers in the world, it's time to shelf the pocket protector, clean the stacks of burnt or blank CDs, get a haircut, stop living on Red Bull and be more social. Try to interact with your teams and peers since a great idea is bad idea if you cannot communicate it and a better idea is to have your good idea validated with others.

SOA from a human perspective

A few months ago I switched jobs. For my new job, I take the Washington Metropolitan Transit Authority Train (Metro) to go to my work location. Everyday it is a different experience. I get on a different train, a different car, sit on a different seat, sit next different people. Sometimes I get an announcement on the Metro that states the the train is delayed for some sort of reason. When my train is delayed then I usually send an email to my boss or call my wife that I will be late for dinner. I also see Metro law enforcement agents, who look for suspicious activities, people or items, get into the Metro and get off the Metro. Why are my observations about Metro interesting?

Well, let's say that I am a XML message and I am sent as a response message from the service providing application. All I, as the XML message, do is sit still in a ESB process (the train) and ride the ESB (Metro equivalent in a SOA environment). I might ride the ESB process with other XML entities who join me during the process. An system owner or an enterprise owner would be comfortable if some sort security mechanisms would be implemented in the ESB where my (as an XML message), fidelity is not altered and copies of myself are covertly created and sent to shady destinations. To address this issue, the system owners should enforce polices of having security agents who act as listeners in the ESB and then call other agents who take care of the corrupt XML individual. As an innocent XML message, I don't want to be tampered with and I would be happy if security controls like agents were implemented on the ESB. I would be upset if my train (the ESB process) is significantly delayed if every security agent is "frisking" every XML message. My owner, the system, would then have to send notices to his customers that I arrived late to the appropriate destination because i was frisked way too many times. Security in the ESB and in the enterprise have to appropriate implemented without significantly slowing down the system.

In airports, we are asked to take off our jackets, shoes, pocket change, cellphones, etc, and etc. This is very inconvenient however we do this because we want to be sure that there are no bad guys on the plane. Even if they are in the plane then they are stripped off of their potential weapons at the airport. This way we know we can reach our destination without much trouble. Using this example, I would like to argue how binary XML is a bad technologies for ESB centric solutions. Since SOA systems are based on distributed archtiectures where trust plays an important role between stakeholders and their systems. If binary XML or even compressed XML is sent across an ESB then rogue agents might sent encrypted viruses and other malicious malware through the ESB. If the ESB decides to decrypt the message then it could possibly infect the whole middleware infrastructure. If it doesn't then the service consumer may be "hosed". This model could bring an enterprise if the malware message sent to several destinations if the message is sent in publish/subscribe model.

Therefore binary XML is a dangerous technology and it should be used in a minimal basis. If you don't think so then please comment to my post.

Enterprise Federated Query is a hoax!

Everyone uses the federated query example as a great application in the Service Oriented Architecture (SOA) paradigm. Via SOA paradigm, the application developers can integrate their client application to numerous services and this will allow the client application users to search and browse various data sources. This sounds great in theory however an enterprise federated query application is not possible. Before we look into why an enterprise federate query application will be a reality, we need to understand what is a federated query application (fqa).

A fqa will allow its users to query multiple data sources at the same time and then the fqa will process the multiple responses into one standardized result set. To the fqa user, it would appear like he or she is hitting one data source. Fqa would be a killer app if it:

• was fast - performance is not an issue
• was very secure - security is not an issue
• was reliable
  
• always returned great results
  

Unfortunately the reality is not nice. Here are the issues:

• High number or infinity number of data sources - The middleware like an Enterprise Service Bus (ESB) takes the fqa user's request and replicate a request for each data source. After the middleware does that, it waits till it gets most if not all of the responses. It then aggregates the responses by removing duplicate results or corrupted results and then forwards the response to the fqa. Imagine the time it is going to take to replicate the requests, wait for the responses, aggregate the responses into one response and then forward it to the fqa. Now imagine if there are five thousand users sending requests at the same time or in a short amount time. The fqa result screen might take a few minutes to render the results. Caching can used to alleviate this problem to a certain degree however it is not possible with realtime or near realtime data.
• Security - Now let us imagine that every data source which is accessed by the fqa requires user credential. This could add time since every data source has to authenticate the user credentials against a data store. This could add increase the response time. The response time is the amount time it takes the fqa user to get the response. Trust based security model would alleviate this issue. The data sources trust that the fqa user has been authenticated by the fqa.
• Data Source variation - Each data source could be different. It could be a Relational Database like Oracle, SQL Server, or DB2; or it something else like:
• Flat file
• Custom Off The Self (COTS) product which has service interfaces
• Object Database
• etc., and etc
The Data source can have specific query language and specific roles. If the Data Source team built services to a common a service contract like a Web Services Description Language (WSDL). This might help however data sources have be optimized to work well with service interfaces. Performance could be an issue

• Result set Aggregation - After the responses are collected, the responses need to be process to remove redundant data set. Computer algorithms need to be developed to aggregate the results and display them.
• Governance - Each of the data sources needs to have appropriate agreements before they can be integrated with the fqa. The agreements can include a Memorandum of Understanding (MOU), Service Level Agreement (SLA), Office Level Agreement (OLA), etc, and etc.

As we can see an enterprise federate query is a large undertaking and it in reality it is not possible because various dependency and it may not be feasible with respect to performance. I built a simple federated query using Yahoo! Pipes but this only works with three data sources.