Saturday, September 7, 2024

digitalEnoch: voice -generated-AI


"Radio station that produces 'Radiolab'"
—That was the clue on the Wednesday, July 24th, 2024, New York Times (NYT) crossword puzzle.  The answer was WNYC, which in my mind stood for W—New York City (NYC).  Intrigued, I went to my podcast app and searched for RadioLab and WNYC, and discovered the Radiolab podcast.  

If you are a science/technology geek and listen to podcasts like me, please check out the Radiolab weekly podcast.  WNYC drops new episodes on Fridays, which have expanded my sphere of knowledge and learning.  For instance, did you know dolphins sleep with half their brain while they use the other half to swim and breathe, or the killer instincts of Argentine ants?  Sometimes, I wish the Argentine men's soccer team had killer instincts against their opponents. 

As someone who works with software, cybersecurity, and data, including artificial intelligence (AI), I find yesterday's RadioLab podcast episode called Shell Game simply fascinating.  It is a good example of how AI voice leveraging generative AI like ChatGPT, Google Gemini, and others are getting sophisticated.  The RadioLab host Latif Nasser interviews Evan Ratliff, a tech journalist who writes about the seedier side of technology.  Evan and his team created a voice AI that sounds just like Evan, and during the episode, Latif plays recorded sessions of Evan's interaction with regular people on the phone.  For instance, in the podcast, I heard the digital Evan interfaces with an online therapist, a $1200/hr attorney, entrepreneurs, etc.  

It made me wonder how a digital version of myself—let’s call it digitalEnoch—would fare.  How would an AI model learn from me and mimic my behavior?  Would digitalEnoch be more analytical or more empathetic? For instance, would digitalEnoch use the word "perfection?"  You will need to listen to the RadioLab podcast to find out about the word "perfection."

In summary, check out this podcast if you want to see how generative voice AI is evolving.  Put on your cybersecurity hat; it's a fascinating future.  I wouldn't say scary because policies, processes and technology are also evolving to put "bad guys" in check.   Here are a couple of links:

As always, please feel free to comment below and follow this blog.

Saturday, August 17, 2024

Give me the list of the top 50 quarterbacks

It's that time of the year again.  I am doing my online research and scouring the internet for tidbits to get ready for this fall and winter's football season.  As a University of Kansas (KU) alum,  I follow my Jayhawks and may go to a game this year.  No, I won't be traveling to Lawrence, Kansas, but I may catch the game in Morgantown, WV.  Thanks to the crazy geographical dispersion of universities in the various significant colleges' sports conferences.  Can we use machine learning and language learning models (LLM) to determine the ideal sports conferences?  We can leverage factors like fan base, endorsements, level of competition, revenue, travel costs, and others to figure out which colleges should reside in which college conferences like the Big Twelve (BIG XII), Big Ten (BIG 10), and others.  

Since this is a technology-centric blog entry, let's get back to the topic at hand.  I get excited about fantasy football every fall, and the pinnacle of any fantasy sports league is the player draft.  I have also been fascinated by how good these LLMs are at meeting my fantasy sports needs.  As someone who has played fantasy sports since 1996,  I have trusted sources.  For this blog entry,  I am going to use the rankings from Yahoo!  Fantasy Sports.  This is what I did.

Purpose:  Leveraging ChatGPT, Google Gemini Advanced, Meta AI and MSN Copilot chatbots, I asked them to give the top 50 quarterbacks for this season.   

Scope:  I focused on NFL quarterbacks because only 32 quarterbacks are playing any given week.  Unlike kickers, quarterbacks have a significant impact on a football game.  

Language Learning Models Used:

AI Vendor LLM
OpenAI Chat GPT GPT-4o
Google Gemini Gemini 1.5 Pro
Meta AI Llama 3.
MSN CoPilot Unknown

The prompt I used for the four sites was:
"Give me the list of the top 50 quarterbacks for my fantasy American football league.  Here is the scoring key: 
Offense League                              Value (points)      
Passing Yards:                                  25 yards per point; 5 points at 350 yards 
Passing Touchdowns:                      
Interceptions:                                  -1 
Rushing Yards:                                 10 yards per point; 5 points at 200 yards 
Rushing Touchdowns:                      6 
Receptions:                                       1
Receiving Yards:                              10 yards per point; 5 points at 200 yards 
Receiving Touchdowns:                  
Return Touchdowns:                        6 
2-Point Conversions:                      
Fumbles Lost:                                 -2 
Offensive Fumble Return TD:         6"

I compared the LLM results with Yahoo!  Fantasy Sports' top 50 quarterbacks (QB).  As of August 16th, 2024, here is the list:
  1. Josh Allen - Buffalo Bills
  2. Jalen Hurts - Philadelphia Eagles
  3. Lamar Jackson - Baltimore Ravens
  4. Patrick Mahomes - Kansas City Chiefs
  5. Anthony Richardson - Indianapolis Colts
  6. C.J. Stroud - Houston Texans
  7. Kyle Murray - Arizona Cardinals
  8. Joe Burrow - Cincinnati Bengals
  9. Dak Prescott - Dallas Cowboys
  10. Jordan Love - Greenbay Packers
  11. Jayden Daniels - Washington Commanders (rookie)
NOTE: The rest of the list is here: Top 50 QBs - 8/16/2024 - Google Sheets

Here is how the various LLM models ranked these quarterbacks.
Yahoo!  Rank Name ChatGPT Meta AI MSN Copilot Google Gemini
1 Josh Allen 1 2 2 1
2 Jalen Hurts 2 4 4 2
3 Lamar Jackson 4 3 3 4
4 Patrick Mahomes 3 1 1 3
5 Trent Richardson 5 5 5 19
6 C.J. Stroud 6 6 6 28
7 Kyle Murray 8 8 8 10
8 Joe Burrow 7 7 7 6
9 Dak Prescott 9 10 10 14
10 Jordan Love 11 9 9 21
11 Jayden Daniels 17 11 11 NOT LISTED

It appears Google Gemini's results (https://gemini.google.com/app/1ab3b15202d9e4f1) did not incorporate the 2024 NFL Draft or the changes in free agency.

Unlike Google Gemini,  ChatGPT, Meta AI, and MSN, Copilot aggregated ranking from other sites.  These sources are listed in the various result sets.  Here are the links to the prompts and the appropriate result sets:
The three LLMs didn't do number crunching to generate the results but instead pulled data from other websites and aggregated the data.

Since I was doing a rudimentary analysis of the various LLMs to meet my needs with my upcoming Fantasy Football draft, I didn't write any code but simply leveraged existing algorithms in Microsoft Excel.   

Results:
Here is the summary of the results:

No. Player Teams Depth Yahoo!  Sports AI Rank
1 Josh Allen Buf 1 1 2
2 Jalen Hurts Phi 1 2 4
3 Lamar Jackson Bal 1 3 3
4 Patrick Mahomes KC 1 4 1
5 Anthony Richardson Ind 1 5 5
6 C.J. Stroud Hou 1 6 6
7 Kyle Murray Ari 1 7 8
8 Joe Burrow Cin 1 8 7
9 Dak Prescott Dal 1 9 10
10 Jordan Love GB 1 10 9
11 Jayden Daniels Was 1 11 11
12 Brock Purdy SF 1 12 12
13 Jared Goff Det 1 13 15
14 Tua Tagovailoa Mia 1 14 13
15 Caleb Williams Chi 1 15 16
16 Trevor Lawrence Jax 1 16 14
17 Kirk Cousins Atl 1 17 17
18 Matthew Stafford LAR 1 18 19
19 Justin Herbert LAC 1 19 18
20 Geno Smith Sea 1 20 22
21 Aaron Rodgers NYJ 1 21 23
22 Deshaun Watson Cle 1 22 20
23 Baker Mayfield TB 1 23 21
24 Will Levis Ten 1 24 24
25 Derek Carr NO 1 25 26
26 Taysom Hill NO 26
27 Daniel Jones NYG 1 27 25
28 Bryce Young Car 1 28 28
29 Bo Nix Den 1 29 30
30 Justin Fields Pit 2 30 27
31 Russell Wilson Pit 1 31 29
32 JJ McCarthy Min 1 32 35.5
33 Jacoby Brissett NE 33 33
34 Sam Darnold Min 34 32
35 Gardner Minshew II LV 1 35 34
36 Drake Maye NE 1 36 31

Regarding the AI Rank, I removed the Google Gemini results using the Standard Deviation algorithm STDEV and the Variance algorithm VAR  since they were outliers compared to the other results.  Here is the link to the spreadsheet (ai-fantasyqb.xlsx - Google Sheets)

Conclusion:
I didn't expect the diversity of the results from the four AI engines and how these AI engines leverage third-party content from other respectable websites.  The other thing I didn't expect is that the results generated from their APIs are quite different since they don't pull data from third-party websites.  In summary, I envision using something other than AI and the various respective LLMs to address my fantasy sports needs.  I expect sports vendors like ESPN, Yahoo!, and sporting organizations to probably have sophisticated AI, LLMs, and machine learning technologies, which will be more reliable for at least the next couple of years.

On a personal note,  if you are an aspiring data scientist, don't be discouraged because you don't have a programming background or understand various algorithms like backward propagation neural networks.  You should be curious to learn from the data.

Friday, August 9, 2024

Large Language Models Asserting Themselves


Wow!  My last post on this blog was on July 26, 2020.  It was four years and 14 days ago, prior to President Joe Biden being elected, hoards of disgruntled individuals storming the US Capitol, and ChatGPT asserting itself on the technology world.  

What is ChatGPT?  It's OpenAI's chatbot.  It is based on artificial intelligence, specifically Large Language Models (LLM).  If you ask ChatGPT, "Can you write me a blog entry about ChatGPT and "LM?" it gives me the following answer: "Certainly! Here’s a blog entry about ChatGPT and Large Language Models (LLMs).” Assuming you clicked on the link,  you can see it did my work for you.  Initially, it was great, but the more I played around with it, I realized that, like any other technology,  it has its benefits and drawbacks.

It's great for prototyping and brainstorming.  I, however, would not put merit into developing new products with the appropriate quality measures.  For instance, I created outlines for strategy documents, but the selection and sequencing of the document sections were based on what I was trying to communicate to the audience.  Recently, I began developing a board game for kids, and I wanted to create a quick prototype on the web using ReactJS.  The prototype is to refine the gameplay, and I didn't spend a lot of time architecting the ReactJS application.  Still, I realized that it wasn't easy to customize ChatGPT-generated code. I didn't have a good grasp of the ReactJS fundamentals.  I, therefore, decided to take a brief pause and focus on the fundamentals, including the syntax.  A few days ago, I got selected to be an AI tester for Outlier.ai.  It's fascinating how the backend works, and products like ChatGPT, Google Gemini, and others need refinement.  Since I was introduced to cybersecurity at work around the time of my previous blog post, I am interested in seeing how the cybersecurity landscape will evolve LLMs.   Black hat hackers and other "baddies" are the creative ones since they are constantly exploiting technologies to make a profit.  I am specifically interested in how data, cyber, and IT systems intersect without impacting user experience and optimizing business.  

Sunday, July 26, 2020

Will computers be allowed to do more than "aping"

Yesterday I completed reading, "What Can Bonobos Teach Us About The Nature of Language," in the July-August 2020 edition of the Smithsonian Magazine. Lindsay Stern (TwitterLinkedIn) wrote the article, which is about the Bonobo ape and their ability to grasp complex language concepts using lexigrams. The research was pioneered by Dr. Sue Savage-Rumbaugh (Linkedin). Sue was able to communicate with lexigrams to the Bonobos apes. She would show a banana and point to a specific symbol. Over time, the bonobos mapped the symbol for bananas, apples, and other objects and communicated with Sue and other researchers. Dr. Savage-Rumbaugh was eventually able to communicate complex thoughts using the lexigrams. Dr. Savage-Rumbaugh was eventually replaced due to the close bond she developed with the bonobos. The current researches state that Dr. Savage-Rumbaugh deviated from the scientific rigors in her research and, in many ways, neglected the apes.

In her Smithsonian article, Stern questions if humans don't want to develop a relationship with the bonobos because it makes us less human or the apes more human. The thought is intriguing, and it led me to ponder if humanity will a similar struggle with machines in the near or distant future. After all, we teach computers using deep-learning algorithms to map images, which can be equated to lexigrams, to objects. Unlike animals, machines are predictable and can be harnessed to superhuman tasks like crunching through petabytes of data in a matter of minutes. If a PC or a Mac says, "I don't want to do the task, " then we replace the machinery. If an employee says, " I don't want to do the task," then the employee is disciplined or even fired. Is it a control thing; or is it our biases (acquired or built-in)


As a trained data scientist, I was taught that it is tough to remove the audience's biases. The best a data scientist can do is to provide the data in a consumable format that will enable the audience to get insights and make decisions.   


Organizations and governments fund research to learn about things that can be leveraged to benefit their respective groups and, eventually, humanity. However, these entities will not fund research that questions the underlying belief systems and makes us less human or our research subject or technology more human. Hopefully, you enjoyed the blog entry because the Smithsonian article made me reflect on the work we do and how we contribute and transform the "human race."


Tuesday, February 4, 2020

Surprise, Surprise,...blah tech isn't working

Surprise. Surprise. A few days ago, IBM replaced its CEO Ginni Rometti, with Arvind Krishna. According to Barrons, IBM sales have fallen by 25% in the last eight years, while other tech giants sales are doing well. In the previous eight years, I haven't seen IBM make the rounds to sell their products like DB2, Watson, WebSphere, and others.  They are not marketing their services well.

These days when I think about IBM, I see it as a "blah" technology company that lacks excitement. I used to think of Microsoft the same way. Things changed after Satya Nadella took reigns at Microsoft. They made a few exciting acquisitions that tell me Microsoft (specifically the current CEO) have a vision. These acquisitions diversify their offering of services. Here is the timeline:
  1. Microsoft made its flagship product Microsoft Office accessible on non-Windows devices.
  2. Microsoft purchased Minecraft for $2.5 billion. I thought this was a head-scratcher, but my kids love Minecraft. Right off the bat, Microsoft made inroads with the next generation, which will be earning income in the five to ten years.
  3. Microsoft introduces Windows 10, and it is a solid Operation System with a smooth user experience. When l look at Windows 7 and others, I am reminded of Steve Balmer and his poor image of screaming at developers. Steve Balmer and his version of Microsoft.  Steve is not missed. He is better off running the Los Angeles Clippers.
  4. Microsoft then bought LinkedIn.
  5. Microsoft purchased Github.  As a developer,  I thought it was huge because it showed Microsoft's commitment to the developer community and the open-source community.

When I see IBM,  I don't see any innovation, and it appears to be the "Big Blue Stale Machine."  Is IBM's Watson ready to collect its Social Security checks because it seems to be aging pretty fast?  The company I am really concerned about is Google. Google recently stopped offering its core product Google Search Appliance, and it bundled it with its email.  Gmail is good, but it doesn't provide the email experience of Outlook.  Google's founders stepped away from leadership and gave it an up-and-coming CEO  Sundar Pichai.  The sudden disappearing act by founding computer scientists Larry Page and Sergey Bring doesn't build my confidence in Google.  Memes like this one below or The Wallstreet Journal article will not help Google's image.
For Google to build their trust with me, I want to see Google's founders take a more active role in the technical community.  I would also like to see user-centric innovation.  Having the most robust and secure cloud in the market is not appealing to me because they don't market it, and their tools are not user-friendly.  It's time for Larry and Sergi to stop playing in their lab and step up with the big boys like Satya Nadella, MarkZuckerburg, Reed Hastings, Tim Cook, and others.

Speaking about Tim Cook, who is the CEO of Apple Inc.  He has not done anything earth-shattering, but he has kept Apple with the times.  The launch of Apple TV is great, but I would like to see more innovation come out of Apple. Shrewd CEOs are strategic, operations focused, and can be ruthless.  Very few leaders are also innovative.  I see Steve Jobs in this category.  As you may know, the US government is trying to wrestle with Apple to get into its devices via a backdoor.  It will be interesting to see if Apple will every give-in.

Unlike Apple, Mark Zuckerburg, and his company Facebook seem to give in to external pressures.  According to Facebook, they are going to let politicians publish political ads.  Facebook is trying to diversify its portfolio of products, but I don't know how they can much progress since I don't trust.  It's going to be hard for Facebook to get back in the game until they gain back the public trust. It can be done.

I am interested to see how Netflix will evolve. Scott Hastings and his team revolutionized streaming digital content to mass audiences. Netflix has created a great market for actors, actresses, producers, directors, and other auxiliary roles for making movies, documentaries, and tv shows. With Disney+, Peacock TV, CBS Access, HBO, and others are trying to catch up to Netflix. According to my graphic, Netflix needs to move into gaming and devices to differentiate itself from other sorts of offerings.

I believe Amazon has reached its potential with e-commerce, and I like the way they are going with Amazon cloud and digital content. Amazon will evolve and begin to the left on my image via Amazon Web Services (AWS).

In summary,  big tech companies need to offer a diverse set of services which gives them the flexibility to grow and help them to minimize the risk.

Saturday, January 18, 2020

ABCDEF - Astros, Baseball, Cheating, Data, Ethics, and Finances

“…Luhnow is widely considered to be one of the most successful baseball executives of his generation, credited with ushering in the second “analytics” revolution in baseball and rebuilding the Houston Astros into a perennial Postseason contender. But while no one can dispute that Luhnow’s baseball operations department is an industry leader in its analytics, it is very clear to me that the culture of the baseball operations department, manifesting itself in the way its employees are treated, its relations with other Clubs, and its relations with the media and external stakeholders, has been very problematic. At least in my view, the baseball operations department’s insular culture – one that valued and rewarded results over other considerations, combined with a staff of individuals who often lacked direction or sufficient oversight…”
-          Excerpt from the Statement of the Commissioner

If you are a baseball fan, then your world got rocked by Major LeagueBaseball’s (MLB) Commissioner Rob Manfred’s penalties on the Houston Astros General Manager (GM) Jeff Luhnow and manager A J Hinch.  They are banned from MLB baseball for a year without pay.  Subsequently the owner of Houston Astros Jim Crane fired both of them.  Jeff and A J were fined because they oversaw the 2017 Houston Astros baseball team, which cheated in baseball games using technology.  The following few days after the firings, Alex Cora,  the Boston Red Sox manager, and Carlos Beltran, the New York Mets manager, stepped down.  Both of them were mentioned in Rob Manfred's Statement of the Commissioner report. Alex was bench coach of the 2017 Houston Astros, and Carlos Beltran was a player on the 2017 Houston Astros team.

The 2017 Houston Astros used technology to steal the opposing team’s catcher’s signs.  The catcher’s signs tell the pitcher what pitch the pitcher will throw.  If the batter knows the pitch that he will see, then the batter has a better chance of hitting the ball.  It alters the odds provides an edge to the batter.


If you don't know much about the game baseball, here is a video that provides the basics of the game.

Other than baseball pundits and historians debate if the penalities on Jeff and AJ were harsh enough and the adverse impact to the game,  the following statements in the commissioner's report caught my attention,
"Luhnow’s baseball operations department is an industry leader in its analytics," and
"the baseball operations department’s insular culture – one that valued and rewarded results over other considerations."  

Ideally, in any data and analytics organization, decisions are based on data and not on emotions and "gut feeling."  The Houston Astros focused on the data and results and forgot their mission, which is to play honest baseball and compete.   MLB players, managers, and teams are tempted to cheat because they can win, which in turn leads to more lucrative financial contracts.

Speaking about money. Several years ago, I distinctly remembered listening to a Fresh Air episode on the National Public Radio (NPR).  In the show, the Fresh Air host interviews Michael Lewis about his book Flash Boys.  The show caught my attention.   The Fresh Air website states the following,
"Flash Boys is about the form of computerized transactions known as high-frequency trading, in which the fastest computers with the highest connection speeds get the information first, and make the trade before anyone else can. A millisecond — even a nanosecond — can make all the difference between how much money is made or lost on any transaction."

With machines making trade decisions in a few seconds,  investment banks, hedge funds are exploiting these types of legal cheating using technology.

I believe US law enforcement agencies like the Federal Bureau of Investigation (FBI), US Securities and Exchange Commission (SEC), and others are looking into these types of legal cheating using technology.

Major League Baseball (MLB) punished the cheating team, but I believe they are of the tip of the iceberg.  With financial institutions using machine learning, artificial intelligence, and other technologies to legally cheat, we, as a society, need to develop policies based on data ethics.

I, however, haven't read much about data ethics in any major publications like Harvard Business ReviewMedium, Gartner, Forrester, and others.  These publications still promote the objectivity of data-driven decisions since it enables organizations to make ideal decisions.  With high-frequency trading,  stealing baseball signs, and others,  I would like to read about how businesses and organizations are promoting data ethics with their workforce.  The domain of Data Ethics should include data stealing,  skewing the data to achieve desirable results, data sharing, speedy data access, and more.

If we don't have penalties for cheating with technology and data, then we may be cheated out of our money.  Companies sharing our personal data for monetary gains is just the being.  Folks need to know what is right and wrong when these individuals work with data and analytics.  We cannot afford to have our trusted institutions to be like the 2017 Houston Astros.

NOTE: The ABCDEF image is from https://www.123rf.com/photo_10483641_flip-clock-letters-a-b-c-d-e-f.html.

Friday, January 3, 2020

Thank you for the 2010s and looking forward to the 2020s.

Happy New Year!!!  As we look forward to the new decade,  we should reflect on the past decade.  

Content-driven services
A lot happened in the 2010s,  Netflix, Amazon Prime, ESPN 3 made streaming content "cool."  I went this route because my DVDs were getting scratched, and I was not happy with my cable service.  Overall I saved money, and I was pleased with its flexibility. I could watch content on my handheld devices.  

Prediction for the 2020s:  

  • I envision these types of streams will continue to grow, and the cable TV will eventually disappear.  Folks will pay for recorded and live content rather than specific tv channels.  I would rather directly pay the NBA, NFL, MLB, and NHL rather than go through ESPN and FOX sports.  
  • I envision broker companies that will focus on aligning users, content providers, and marketing companies.  The broker companies will develop user profiles based on patterns.  They will use  AI and machine learning to build and improve user profiles.  These profiles will then leveraged to provided target adds.  Amazon is already taking steps to go this route via Amazon Channels[1].  
Technical Debt

Currently, Chief Information Officers (CIO) are struggling with aging infrastructure and the lack of funds to upgrade the infrastructure, which includes network infrastructure, databases, applications, and other IT-related components.  Hackernoon.com does a great job of defining Technical Debt[2], and Technical Debt is one of the significant causes of cybersecurity vulnerabilities.

Prediction for the 2020s:  
  • Companies will be forced to address this problem in the 2020s because of security vulnerabilities that be in exposed in the old infrastructure.
  • I envision start-up companies that will specialize in upgrading infrastructure, which includes moving digital assets (e.g., applications, data, and others) to the cloud. These companies will bring processes like DevSecOps[3] and the associated tools to the project.
  • These companies will need to provide a holistic approach in assessing the issues, developing a strategy, and executing the plan. 
Leverage platform services

In the 2010s,  companies recognized that they need more enterprise IT services. Email isn't the only enterprise service. Services like Identity and Access Management (IAM) services, enterprise search,  document management, and others need to be considered as well. Companies bought the infrastructure and hired the staff to develop the service and maintain it.  Unfortunately, a document management SME doesn't have the bandwidth to understand the rapidly evolving cybersecurity attacks.  This leaves the businesses behind the eight-ball when it comes to sustaining an enterprise service and be vigilant of cyber threats.

Prediction for the 2020s:  
  • Large vendors like Google, Facebook, Microsoft, and others will offer cloud-based enterprise services.  These services will have simple and clean interfaces.  By having a clean and straightforward interface,  the potential targets for a cyber attack are reduced. The current set of examples of cloud-based enterprise services are the Google Identity Platform [4], Tableau Online [5], and others. 
  • I envision IT groups focus on training their staff on how to integrate their business applications with their enterprise services.  

CXO roles

Currently, the Chief Information Officer (CIO) is the chief IT officer in any business organization.  Typically the Chief Information Security Officer (CISO) and the Chief Data Officer (CDO) report to the CIO.  I envision these roles will change in the future.

Prediction for the 2020s:
  • Due to the continuous threat of cyber attacks, the role of the CISO will be elevated in the business organization.  The CISO will be equivalent to the CIO.  
  • The job of the CDO will continue to be a tough one[6].  The CDO role will not be funded well and may not have a lot of authority.  The CDO will need to please two masters, the CDO and CISO.
  • The CDO role will follow a similar pattern to the Gartner Hype Cycle [7], where the hype will go down because the CDO role is not an operations role.  The CDO will be on the same level as the Chief Technology Officer (CTO) or lower.
  • The Cloud Architect will be elevated to the C-Suite due to the complexity of the cloud and the cybersecurity threats. On December 30th, 2019, The Wallstreet Journal published an article, Ghosts in the Clouds: Inside China’s Major Corporate Hack[8], which discusses how the Chinese hackers went in through the cloud providers.

[1] Casey, H., and Reisinger, C. (2019, May 13). What Is Amazon Channels and Is it Worth It? Tom's Guide. Retrieved from https://www.tomsguide.com/us/amazon-channels-faq,review-4125.html.
[2] Hackernoon.com. (2018, Jan. 25). There are 3 main types of technical debt. Here’s how to manage them. Retrieved from https://hackernoon.com/there-are-3-main-types-of-technical-debt-heres-how-to-manage-them-4a3328a4c50c.
[3] RedHat, Inc., (n.d.). What is DevSecOps?  Retrieved from https://www.redhat.com/en/topics/devops/what-is-devsecops.
[4] Google, LLC (n.d.). Google Identity Platform. Retrieved from https://developers.google.com/identity
[5] Tableau, LLC (n.d.). Tableau Online. Retrieved from https://www.tableau.com/products/cloud-bi.
[6] Bennett, Jo. (2016, Apr. 11). How chief data officers can tackle formidable roadblocks, including people, culture, and internal resistance. Smarter With Gartner. Gartner, Inc.  Retrieved from https://www.gartner.com/smarterwithgartner/half-of-cdos-succeed/.
[7] Gartner, Inc. (n.d.). Gartner Hype Cycle.  Retrieved from https://www.gartner.com/en/research/methodologies/gartner-hype-cycle.
[8] The Wallstreet Journal (2019, Dec. 30). Ghosts in the Clouds: Inside China’s Major Corporate Hack. Retrieved from https://www.wsj.com/articles/ghosts-in-the-clouds-inside-chinas-major-corporate-hack-11577729061.