Hack the LIME - the story about open-source AI

Leave it to a lawyer to spoil the fun!  Andrew Burt, a lawyer, and writer, recently wrote an article for the Harvard Business Review, which discusses the risks of open-source Artifical Intelligence (AI) models   The HBR post, "The AI Transparency Paradox," explains the dangers of making artificial intelligence models transparent.  On the one hand, businesses want to know how companies engineered their AI algorithms.  Is the AI algorithm too sexist, racist, and just not good?  For instance, in James Vincent's article, Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech, Google "fixed" its 2015 deep learning algorithm, which labeled humans as monkeys. I am sure the appropriate stakeholders at Google wanted to know how the algorithm was written and how can the developer fixed the inappropriate bias in the algorithm.

On the one hand, if Google opened its AI code to the open-source community, then the AI algorithms could be significantly improved.   On the other hand, as (our no-fun lawyer friend), Andrew Burt cautions businesses that they would be vulnerable if they opened their AI algorithms to the world, which includes bad actors like hostile governments.  Andrew uses the research paper titled, “Why Should I Trust You?” Explaining the Predictions of Any Classifier.  The paper is about the Local Interpretable Model-Agnostic Explanations (LIME) algorithm.  Andrew then talks about a research paper. How can we fool LIME and SHAP? Adversarial Attacks on Post hoc Explanation Methods,  which was published in November 2019.  The document discusses a "novel scaffolding technique," which can be used to hack the LIME algorithm.  Is there a return on investment if companies made their algorithms more transparent?

Folks may argue that Google successfully made its TensorFlow machine learning (ML) platform open source without much impact.  Before the TensorFlow platform question can be answered,  the following terms need to be defined:

• Machine Learning;
• Deep Learning; and 
• Artificial Intelligence.

According to Geeks for Geeks portal, "Machine Learning is a technique of parsing data, learn from that data, and then apply what they have learned to make an informed decision." Machine Learning includes supervised training where the machine builds the model on most of the data (70% to 90% of the complete dataset) and then test the data for accuracy with the remaining portion of the data.  Unsupervised training involves machine learning from the full dataset.  There is no testing of the data.

Deep Learning algorithms are a specific subset of Machine Learning algorithms, which are primarily composed of neural networks. A common use of deep learning algorithms is for image recognization.

Gartner defines Artificial intelligence (AI) is the application of "...advanced analysis and logic-based techniques, including machine learning, to interpret events, support and automate decisions, and take actions."

Gartner recommends several open-source Data Science and ML platforms, including TensorFlow, to develop ML-based solutions.    Does this mean Mr. Burt is incorrect?

Andrew Burt is correct because comprised algorithms, which are the core of all AI solutions, will cause businesses to make poor decisions and ultimately discredit the AI solution. The underlying technology, which enables to invoke the algorithm, pull the data, process the data, and reports the outcomes, can be open source.  The other critical piece of any AI solution is the data itself.   Sensitive data is never unless asked by law enforcement or regulatory agency.

In conclusion,  Andrew Burt is correct by bringing up valid points, but at the same time, the AI field is still pretty young.  I am a strong believer in open-source software, and I believe we will have open-source AI models available.  We will also have AI models that will validate other models using technologies like blockchain technologies.

NOTE: The video below is about the LIME algorithm, and please keep in mind researchers recently hacked this algorithm.

Netflix Phenomenon and Mobility

Three weeks ago, I attended a two day workshop called the mGov Strategy.  This workshop's purpose is to provide input to OMB's mobility strategy for the US Federal Government.  Steve VanRoekel, OMB CIO, sponsored this workshop.  US civil servants from various lines of government joined the workshop. OMB split the working group into five sub-groups.  The sub-groups were:

1. Acquisition - How can US government address acquisition of mobile technologies and services? Can the US government streamline the acquisition process.
2. Security - How to safeguard government information and technologies from hackers.
3. Privacy - How to protect mobile user information from inappropriate use especially when they interface with US government mobile sites and apps.
4. Citizen apps - How to develop a mobile presence to engage US government's biggest customer US citizen.
5. Infrastructure - How can US government address the evolution of mobile technologies and associated technologies like cloud computing, social computing and others.

As the member of the security sub-group, we discussed several policy and technical approaches.  The thing that caught my eye and basically sums up any future technical advancement is the ability to do use any application from anywhere and anytime.  I call this the "Netflix phenomenon"

Even though Reed Hastings, Netflix CEO, won't win the CEO of the year award, I still give him credit in taking the movie watching experience from a cinema theater to any possible device which is accredited by Netflix.  I admit that Google introduced this feature with YouTube however Netflix took it to a new level.  I can now start a movie via  my laptop, pause it and then resume it on my iPad.  I like this DirecTV commerical which captures what I am talking about.


To develop this type of an IT service, enterprises need to invest in the following technologies and architectures like:

• Cloud computing - IT departments need to centralize their business applications and act as cloud brokers to outsource some of their applications to third party clouds like Amazon EC2, Google Cloud, Rackspace and others.  I believe unless OMB makes significant investments in IT infrastructure, agencies will have to act as cloud brokers. It's a cost effective mechanism.
• Smarter Pipes - Where is Mario when you need him?  With all of the data and information streaming back and forth between clouds, user devices, government needs to influence how IT networks should evolve.  Since mobile users are constantly starving for the fastest network, vendors have to realize that simply scaling up the networks is not a sustainable model.  Vendors and research institutes need to look at how data should traverse the network and optimize it.  A good example is that vendors need to develop information caching mechanisms at the network level.  
• Smarter security - One of the best phrases used by the mobile users in the government space is, "brick". Users can call and email on a brick but nothing else.  Security personnel should realize that clamping everything defeats the purpose.  IT risk management should be a key in developing a smarter security posture. Single sign on is key as well. No one wants to sign on with multiple usernames and passwords to do their work.
• Usability - One of the best parts of using Netflix is how intuitive the user interface is.  Ease of use is a key phrase to describe Netflix's user interface.  We need to identify and prioritize what functionality is needed or desired on a mobile app.
• Flexibility - Use sound architecture principles like loose coupling, simple interfaces and architectures.  Simpler is better.  
• Standards based architecture -  Eventhough there is an over abundance of  standards especially XML (frankly I am sick of how folks are misusing it), we still need to emphasize it and design appropriately.  Having a 50MB XML payload in SOA enabled information exchange is NOT smart architecture.  I am not going to expound on the 50MB XML example since it is aggravating.

Even though Reed Hastings didn't make alot friends with Qwikster or jacking up the monthly Netflix fees,  he did build a pretty cool service called Netflix streaming.  As I write this blog, my youngest son is watching Power Rangers in Space via the WII and my oldest two are watching a Dreamworks movie via the PS3.  My third one is having fun the old fashioned way. She is attending a birthday party. Thanks Reed and now bring down the monthly fees.