Reading, listening

• 2023-05-26 Gorilla: Large Language Model Connected with Massive APIs
  • Gorilla can use used as a language aware LLM that delegates that outputs an API, which can be used to retrieve results from another LLM
• 2023-05-26 The False Promise of Imitating Proprietary LLMs:
  • A relatively smaller LM was trained on ChatGPT prompt data (key = prompt, value = chatgpt output). For generic questions the new LM behaved really well. But when test prompts were more specific, there wasn’t much difference and in many cases the quality degraded.
  • The imitation models learn ‘Style’ and not ‘Content’ of ChatGPT responses
  • So we’re not at a point where we can have an exponential increase in the quality of LLMs by using the prompt data for a base LM, to train a new LM. And then using the prompt data for the second LM to train a 3rd LM .. and so on.
• 2023-05-26 Gravitational-wave detector LIGO is back — and can now spot more colliding black holes than ever
  • the detector was down for 3 years and is back up again. The upgrades now allows it to detect a merging blackhole every 2-3 days instead of the previous 1 week gap.
• 2023-05-26 How AI Is Catapulting Nvidia Toward the $1 Trillion Club
  • With the hardware needs for LLMs, Nvidia stock is going up quite a bit. Reminds me of friends who had bought Nvidia’s stock at $25/share several years ago and had made a ton of money after it had spiked.
• 2023-05-27 How the Lemon was Invented
  • pomelo + mandarin = bitter orange
  • bitter orange + citron (has thick skin) = lemon
  • in 1700s, when physicians realized that lemon juice prevents scurvy, it started getting planted across major shipping routes
• 2023-05-30 Superintelligence: The Idea That Eats Smart People
  • Maciej Ceglowski - Superintelligence: The Idea That Eats Smart People (Keynote)
  • The theory that Superintelligence / AGI will take over the world very soon isn’t correct. It has too many false assumptions.
  • AI is like Alchemy where we have some hints of what could be going on in reality but are far away from understanding it.
  • Alchemists would compare a rock and a log of wood and think that they’re made of the same substance but in reality wood is much more complex than stone. The same way, we’re comparing AI with human intelligence / sentience and thinking they might be very similar. But we don’t really know how different they are or could be.
• 2023-06-01 OpenAI’s plans according to Sam Altman
  • OpenAI is currently constrained by GPU availability. No wonder the stock price of Nvidia has sky-rocketed
  • OpenAI doesn’t plan to launch any additional products using ChatGPT. So developers can be less worried about competing products in the near future.
• 2023-06-03 My Business Card Runs Linux • &> /dev/null
  • Wow, such a cool idea! This is not just a USB drive with the person’s resume .. this is the entire Linux running on the business card. (Image source: https://www.thirtythreeforty.net)
• 2023-06-06 Episode 367 – Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast
  • RLHF (Reinforcement Learning with Human Feedback) makes the model remarkably better
  • After consuming all the human knowledge, GPT seems like developing a ‘reasoning’ system.
  • GPT4 might be the most complex piece of software humanity has ever built
  • Satya Nadella is both - a very strong visionary and makes long duration correct calls. But is also a very effective hands-on executive too.
  • Sam Altman - It’s way too tempting to take advice from other people…. I mostly got what I wanted by ignoring advice.