Give me the list of the top 50 quarterbacks

It's that time of the year again.  I am doing my online research and scouring the internet for tidbits to get ready for this fall and winter's football season.  As a University of Kansas (KU) alum,  I follow my Jayhawks and may go to a game this year.  No, I won't be traveling to Lawrence, Kansas, but I may catch the game in Morgantown, WV.  Thanks to the crazy geographical dispersion of universities in the various significant colleges' sports conferences.  Can we use machine learning and language learning models (LLM) to determine the ideal sports conferences?  We can leverage factors like fan base, endorsements, level of competition, revenue, travel costs, and others to figure out which colleges should reside in which college conferences like the Big Twelve (BIG XII), Big Ten (BIG 10), and others.  

Since this is a technology-centric blog entry, let's get back to the topic at hand.  I get excited about fantasy football every fall, and the pinnacle of any fantasy sports league is the player draft.  I have also been fascinated by how good these LLMs are at meeting my fantasy sports needs.  As someone who has played fantasy sports since 1996,  I have trusted sources.  For this blog entry,  I am going to use the rankings from Yahoo!  Fantasy Sports.  This is what I did.

Purpose:  Leveraging ChatGPT, Google Gemini Advanced, Meta AI and MSN Copilot chatbots, I asked them to give the top 50 quarterbacks for this season.   

Scope:  I focused on NFL quarterbacks because only 32 quarterbacks are playing any given week.  Unlike kickers, quarterbacks have a significant impact on a football game.  

Language Learning Models Used:

AI Vendor	LLM
OpenAI Chat GPT	GPT-4o
Google Gemini	Gemini 1.5 Pro
Meta AI	Llama 3.
MSN CoPilot	Unknown

The prompt I used for the four sites was:

"Give me the list of the top 50 quarterbacks for my fantasy American football league.  Here is the scoring key: 

Offense League                              Value (points)      

Passing Yards:                                  25 yards per point; 5 points at 350 yards 

Passing Touchdowns:                       6 

Interceptions:                                  -1 

Rushing Yards:                                 10 yards per point; 5 points at 200 yards 

Rushing Touchdowns:                      6 

Receptions:                                       1

Receiving Yards:                              10 yards per point; 5 points at 200 yards 

Receiving Touchdowns:                   6 

Return Touchdowns:                        6 

2-Point Conversions:                       2 

Fumbles Lost:                                 -2 

Offensive Fumble Return TD:         6"

I compared the LLM results with Yahoo!  Fantasy Sports' top 50 quarterbacks (QB).  As of August 16th, 2024, here is the list:

1. Josh Allen - Buffalo Bills
2. Jalen Hurts - Philadelphia Eagles
3. Lamar Jackson - Baltimore Ravens
4. Patrick Mahomes - Kansas City Chiefs
5. Anthony Richardson - Indianapolis Colts
6. C.J. Stroud - Houston Texans
7. Kyle Murray - Arizona Cardinals
8. Joe Burrow - Cincinnati Bengals
9. Dak Prescott - Dallas Cowboys
10. Jordan Love - Greenbay Packers
11. Jayden Daniels - Washington Commanders (rookie)

NOTE: The rest of the list is here: Top 50 QBs - 8/16/2024 - Google Sheets

Here is how the various LLM models ranked these quarterbacks.

Yahoo!  Rank	Name	ChatGPT	Meta AI	MSN Copilot	Google Gemini
1	Josh Allen	1	2	2	1
2	Jalen Hurts	2	4	4	2
3	Lamar Jackson	4	3	3	4
4	Patrick Mahomes	3	1	1	3
5	Trent Richardson	5	5	5	19
6	C.J. Stroud	6	6	6	28
7	Kyle Murray	8	8	8	10
8	Joe Burrow	7	7	7	6
9	Dak Prescott	9	10	10	14
10	Jordan Love	11	9	9	21
11	Jayden Daniels	17	11	11	NOT LISTED

It appears Google Gemini's results (https://gemini.google.com/app/1ab3b15202d9e4f1) did not incorporate the 2024 NFL Draft or the changes in free agency.

Unlike Google Gemini,  ChatGPT, Meta AI, and MSN, Copilot aggregated ranking from other sites.  These sources are listed in the various result sets.  Here are the links to the prompts and the appropriate result sets:

• ChatGPT - https://chatgpt.com/c/7d1ef462-f5b1-44fe-9621-a53e775c0a83
• Meta AI - https://www.meta.ai/c/8a581989-559e-497c-8063-280cd5cf2254
• MSN Copilot - https://copilot.microsoft.com/sl/in1H9YyMRZQ

The three LLMs didn't do number crunching to generate the results but instead pulled data from other websites and aggregated the data.

Since I was doing a rudimentary analysis of the various LLMs to meet my needs with my upcoming Fantasy Football draft, I didn't write any code but simply leveraged existing algorithms in Microsoft Excel.   

Results:

Here is the summary of the results:

No.	Player	Teams	Depth	Yahoo!  Sports	AI Rank
1	Josh Allen	Buf	1	1	2
2	Jalen Hurts	Phi	1	2	4
3	Lamar Jackson	Bal	1	3	3
4	Patrick Mahomes	KC	1	4	1
5	Anthony Richardson	Ind	1	5	5
6	C.J. Stroud	Hou	1	6	6
7	Kyle Murray	Ari	1	7	8
8	Joe Burrow	Cin	1	8	7
9	Dak Prescott	Dal	1	9	10
10	Jordan Love	GB	1	10	9
11	Jayden Daniels	Was	1	11	11
12	Brock Purdy	SF	1	12	12
13	Jared Goff	Det	1	13	15
14	Tua Tagovailoa	Mia	1	14	13
15	Caleb Williams	Chi	1	15	16
16	Trevor Lawrence	Jax	1	16	14
17	Kirk Cousins	Atl	1	17	17
18	Matthew Stafford	LAR	1	18	19
19	Justin Herbert	LAC	1	19	18
20	Geno Smith	Sea	1	20	22
21	Aaron Rodgers	NYJ	1	21	23
22	Deshaun Watson	Cle	1	22	20
23	Baker Mayfield	TB	1	23	21
24	Will Levis	Ten	1	24	24
25	Derek Carr	NO	1	25	26
26	Taysom Hill	NO		26
27	Daniel Jones	NYG	1	27	25
28	Bryce Young	Car	1	28	28
29	Bo Nix	Den	1	29	30
30	Justin Fields	Pit	2	30	27
31	Russell Wilson	Pit	1	31	29
32	JJ McCarthy	Min	1	32	35.5
33	Jacoby Brissett	NE		33	33
34	Sam Darnold	Min		34	32
35	Gardner Minshew II	LV	1	35	34
36	Drake Maye	NE	1	36	31

Regarding the AI Rank, I removed the Google Gemini results using the Standard Deviation algorithm STDEV and the Variance algorithm VAR  since they were outliers compared to the other results.  Here is the link to the spreadsheet (ai-fantasyqb.xlsx - Google Sheets)

Conclusion:

I didn't expect the diversity of the results from the four AI engines and how these AI engines leverage third-party content from other respectable websites.  The other thing I didn't expect is that the results generated from their APIs are quite different since they don't pull data from third-party websites.  In summary, I envision using something other than AI and the various respective LLMs to address my fantasy sports needs.  I expect sports vendors like ESPN, Yahoo!, and sporting organizations to probably have sophisticated AI, LLMs, and machine learning technologies, which will be more reliable for at least the next couple of years.

On a personal note,  if you are an aspiring data scientist, don't be discouraged because you don't have a programming background or understand various algorithms like backward propagation neural networks.  You should be curious to learn from the data.

Large Language Models Asserting Themselves

Wow!  My last post on this blog was on July 26, 2020.  It was four years and 14 days ago, prior to President Joe Biden being elected, hoards of disgruntled individuals storming the US Capitol, and ChatGPT asserting itself on the technology world.  

What is ChatGPT?  It's OpenAI's chatbot.  It is based on artificial intelligence, specifically Large Language Models (LLM).  If you ask ChatGPT, "Can you write me a blog entry about ChatGPT and "LM?" it gives me the following answer: "Certainly! Here’s a blog entry about ChatGPT and Large Language Models (LLMs).” Assuming you clicked on the link,  you can see it did my work for you.  Initially, it was great, but the more I played around with it, I realized that, like any other technology,  it has its benefits and drawbacks.

It's great for prototyping and brainstorming.  I, however, would not put merit into developing new products with the appropriate quality measures.  For instance, I created outlines for strategy documents, but the selection and sequencing of the document sections were based on what I was trying to communicate to the audience.  Recently, I began developing a board game for kids, and I wanted to create a quick prototype on the web using ReactJS.  The prototype is to refine the gameplay, and I didn't spend a lot of time architecting the ReactJS application.  Still, I realized that it wasn't easy to customize ChatGPT-generated code. I didn't have a good grasp of the ReactJS fundamentals.  I, therefore, decided to take a brief pause and focus on the fundamentals, including the syntax.  A few days ago, I got selected to be an AI tester for Outlier.ai.  It's fascinating how the backend works, and products like ChatGPT, Google Gemini, and others need refinement.  Since I was introduced to cybersecurity at work around the time of my previous blog post, I am interested in seeing how the cybersecurity landscape will evolve LLMs.   Black hat hackers and other "baddies" are the creative ones since they are constantly exploiting technologies to make a profit.  I am specifically interested in how data, cyber, and IT systems intersect without impacting user experience and optimizing business.