AI is appearing in seemingly every corner of modern life, from music and media to business and productivity, even dating. There’s so much it can be hard to keep up — so read on to find out everything from the latest big developments to the terms and companies you need to know in order to stay current in this fast-moving field.
To begin with, let’s just make sure we’re all on the same page: what is AI?
Artificial intelligence, also called machine learning, is a kind of software system based on neural networks, a technique that was actually pioneered decades ago but very recently has blossomed thanks to powerful new computing resources. AI has enabled effective voice and image recognition, as well as the ability to generate synthetic imagery and speech. And researchers are hard at work making it possible for an AI to browse the web, book tickets, tweak recipes and more.
Oh, but if you’re worried about a Matrix-type rise of the machines — don’t be. We’ll talk about that later!
Our guide to AI has three main parts, each of which we will update regularly and can be read in any order:
- First, the most fundamental concepts you need to know as well as more recently important ones.
- Next, an overview of the major players in AI and why they matter.
- And last, a curated list of the recent headlines and developments that you should be aware of.
By the end of this article you’ll be about as up to date as anyone can hope to be these days. We will also be updating and expanding it as we press further into the age of AI.
AI 101
Image Credits: Andrii Shyp / Getty Images
One of the wild things about AI is that although the core concepts date back more than 50 years, few of them were familiar to even the tech-savvy before very recently. So if you feel lost, don’t worry — everyone is.
And one thing we want to make clear up front: Although it’s called “artificial intelligence,” that term is a little misleading. There’s no one definition of intelligence out there, but what these systems do is definitely closer to calculators than brains. The input and output of this calculator is just a lot more flexible. You might think of artificial intelligence like artificial coconut — it’s imitation intelligence.
With that said, here are the basic terms you’ll find in any discussion of AI.
Neural network
Our brains are largely made of interconnected cells called neurons, which mesh together to form complex networks that perform tasks and store information. Recreating this amazing system in software has been attempted since the ’60s, but the processing power required wasn’t widely available until 15-20 years ago, when GPUs let digitally defined neural networks flourish. At their heart they are just lots of dots and lines: the dots are data and the lines are statistical relationships between those values. As in the brain, this can create a versatile system that quickly takes an input, passes it through the network and produces an output. This system is called a model.
Model
The model is the actual collection of code that accepts inputs and returns outputs. The similarity in terminology to a statistical model or a modeling system that simulates a complex natural process is not accidental. In AI, model can refer to a complete system like ChatGPT, or pretty much any AI or machine learning construct, whatever it does or produces. Models come in various sizes, meaning both how much storage space they take up and how much computational power they take to run. And these depend on how the model is trained.
Training
To create an AI model, the neural networks making up the base of the system are exposed to a bunch of information in what’s called a dataset or corpus. In doing so, these giant networks create a statistical representation of that data. This training process is the most computation-intensive part, meaning it takes weeks or months (you can kind of go as long as you want) on huge banks of high-powered computers. The reason for this is that not only are the networks complex, but datasets can be extremely large: billions of words or images that must be analyzed and given representation in the giant statistical model. On the other hand, once the model is done cooking it can be much smaller and less demanding when it’s being used, a process called inference.
Image Credits: Google
Inference
When the model is actually doing its job, we call that inference, very much the traditional sense of the word: stating a conclusion by reasoning about available evidence. Of course it is not exactly “reasoning,” but statistically connecting the dots in the data it has ingested and, in effect, predicting the next dot. For instance, saying “Complete the following sequence: red, orange, yellow…” it would find that these words correspond to the beginning of a list it has ingested, the colors of the rainbow, and infers the next item until it has produced the rest of that list. Inference is generally much less computationally costly than training: Think of it like looking through a card catalog rather than assembling it. Big models still have to run on supercomputers and GPUs, but smaller ones can be run on a smartphone or something even simpler.
Generative AI
Everyone is talking about generative AI, and this broad term just means an AI model that produces an original output, like an image or text. Some AIs summarize, some reorganize, some identify, and so on — but an AI that actually generates something (whether or not it “creates” is arguable) is especially popular right now. Just remember that just because an AI generated something, that doesn’t mean it is correct, or even that it reflects reality at all! Only that it didn’t exist before you asked for it, like a story or painting.
Today’s top terms
Beyond the basics, here are the AI terms that are most relevant in mid-2023.
Large language model
The most influential and versatile form of AI available today, large language models are trained on pretty much all the text making up the web and much of English literature. Ingesting all this results in a foundation model (read on) of enormous size. LLMs are able to converse and answer questions in natural language and imitate a variety of styles and types of written documents, as demonstrated by the likes of ChatGPT, Claude and LLaMa. While these models are undeniably impressive, it must be kept in mind that they are still pattern recognition engines, and when they answer it is an attempt to complete a pattern it has identified, whether or not that pattern reflects reality. LLMs frequently hallucinate in their answers, which we will come to shortly.
If you want to learn more about LLMs and ChatGPT, we have a whole separate article on those!
Foundation model
Training a huge model from scratch on huge datasets is costly and complex, and so you don’t want to have to do it any more than you have to. Foundation models are the big from-scratch ones that need supercomputers to run, but they can be trimmed down to fit in smaller containers, usually by reducing the number of parameters. You can think of those as the total dots the model has to work with, and these days it can be in the millions, billions or even trillions.
Fine tuning
A foundation model like GPT-4 is smart, but it’s also a generalist by design — it absorbed everything from Dickens to Wittgenstein to the rules of Dungeons & Dragons, but none of that is helpful if you want it to help you write a cover letter for your resumé. Fortunately, models can be fine tuned by giving them a bit of extra training using a specialized dataset, for instance a few thousand job applications that happen to be laying around. This gives the model a much better sense of how to help you in that domain without throwing away the general knowledge it has collected from the rest of its training data.
Reinforcement learning from human feedback, or RLHF, is a special kind of fine tuning you’ll hear about a lot — it uses data from humans interacting with the LLM to improve its communication skills.
Diffusion
From a paper on an advanced post-diffusion technique, you can see how an image can be reproduced from even very noisy data. Image Credits: OpenAI
Image generation can be done in numerous ways, but by far the most successful as of today is diffusion, which is the technique at the heart of Stable Diffusion, Midjourney and other popular generative AIs. Diffusion models are trained by showing them images that are gradually degraded by adding digital noise until there is nothing left of the original. By observing this, diffusion models learn to do the process in reverse as well, gradually adding detail to pure noise in order to form an arbitrarily defined image. We’re already starting to move beyond this for images, but the technique is reliable and relatively well understood, so don’t expect it to disappear any time soon.
Hallucination
Originally this was a problem of certain imagery in training slipping into unrelated output, such as buildings that seemed to be made of dogs due to an an over-prevalence of dogs in the training set. Now an AI is said to be hallucinating when, because it has insufficient or conflicting data in its training set, it just makes something up.
This can be either an asset or a liability; an AI asked to create original or even derivative art is hallucinating its output; an LLM can be told to write a love poem in the style of Yogi Berra, and it will happily do so — despite such a thing not existing anywhere in its dataset. But it can be an issue when a factual answer is desired; models will confidently present a response that is half real, half hallucination. At present there is no easy way to tell which is which except checking for yourself, because the model itself doesn’t actually know what is “true” or “false,” it is only trying to complete a pattern as best it can.
AGI or strong AI
Artificial General Intelligence, or strong AI, is not really a well-defined concept, but the simplest explanation is that it is an intelligence that is powerful enough not just to do what people do, but learn and improve itself like we do. Some worry that this cycle of learning, integrating those ideas, and then learning and growing faster will be a self-perpetuating one that results in a super-intelligent system that is impossible to restrain or control. Some have even proposed delaying or limiting research to forestall this possibility.
It’s a scary idea, sure, and movies like “The Matrix” and “Terminator” have explored what might happen if AI spirals out of control and attempts to eliminate or enslave humanity. But these stories are not grounded in reality. The appearance of intelligence we see in things like ChatGPT is an impressive act, but has little in common with the abstract reasoning and dynamic multi-domain activity that we associate with “real” intelligence. While it’s near-impossible to predict how things will progress, it may be helpful to think of AGI as something like interstellar space travel: We all understand the concept and are seemingly working toward it, but at the same time we’re incredibly far from achieving anything like it. And due to the immense resources and fundamental scientific advances required, no one is going to just suddenly accomplish it by accident!
AGI is interesting to think about, but there’s no sense borrowing trouble when, as commentators point out, AI is already presenting real and consequential threats today despite, and in fact largely due to, its limitations. No one wants Skynet, but you don’t need a superintelligence armed with nukes to cause real harm: people are losing jobs and falling for hoaxes today. If we can’t solve those problems, what chance do we have against a T-1000?
Top players in AI
OpenAI
Image Credits: Leon Neal / Getty Images
If there’s a household name in AI, it’s this one. OpenAI began as its name suggests, an organization intending to perform research and provide the results more or less openly. It has since restructured as a more traditional for-profit company providing access to its advances language models like ChatGPT through APIs and apps. It’s headed by Sam Altman, a technotopian billionaire who nonetheless has warned of the risks AI could present. OpenAI is the acknowledged leader in LLMs but also performs research in other areas.
Microsoft
As you might expect, Microsoft has done its fair share of work in AI research, but like other companies has more or less failed to turn its experiments into major products. Its smartest move was to invest early in OpenAI, which scored it an exclusive long-term partnership with the company, which now powers its Bing conversational agent. Though its own contributions are smaller and less immediately applicable, the company does have a considerable research presence.
Google
Known for its moonshots, Google somehow missed the boat on AI despite its researchers literally inventing the technique that led directly to today’s AI explosion: the transformer. Now it’s working hard on its own LLMs and other agents, but is clearly playing catch-up after spending most of its time and money over the last decade boosting the outdated “virtual assistant” concept of AI. CEO Sundar Pichai has repeatedly said that the company is aligning itself firmly behind AI in search and productivity.
Anthropic
After OpenAI pivoted away from openness, siblings Dario and Daniela Amodei left it to start Anthropic, intended to fill the role of an open and ethically considerate AI research organization. With the amount of cash they have on hand, they’re a serious rival to OpenAI even if their models, like Claude, aren’t as popular or well-known yet.
Stability
Image Credits: Bryce Durbin / TechCrunch
Controversial but inevitable, Stability represents the “do what thou wilt” open source school of AI implementation, hoovering up everything on the internet and making the generative AI models it trains freely available if you have the hardware to run it. This is very in line with the “information wants to be free” philosophy but has also accelerated ethically dubious projects like generating pornographic imagery and using intellectual property without consent (sometimes at the same time).
Elon Musk
Not one to be left out, Musk has been outspoken about his fears regarding out-of-control AI, as well as some sour grapes after he contributed to OpenAI early on and it went in a direction he didn’t like. While Musk is not an expert on this topic, as usual his antics and commentary do provoke widespread responses (he was a signatory on the above-mentioned “AI pause” letter) and he is attempting to start a research outfit of his own.
Latest stories in AI
The U.S. Department of Commerce could prohibit shipments of chips from manufacturers including Nvidia to customers in China as soon as early next month (July).
The latest move to weigh additional restrictions on AI chip export to China is part of the U.S.’s broader strategy to limit China’s progress in AI, particularly in the military sphere. However, these measures are also having an adverse impact on the commercial AI sector in China, where many firms operate with teams that span both the U.S. and China.
ChatGPT Plus subscribers can now access a new feature on the ChatGPT app called Browsing to have ChatGPT search Bing for answers to prompts or questions. OpenAI says that the Browsing feature is particularly useful for queries relating to current events and other information that “extend[s] beyond [ChatGPT’s] original training data.” When Browsing is disabled, ChatGPT’s knowledge cuts off in 2021.
If a musician’s AI-assisted composition is to be eligible for a Grammy, they’ll need to make sure that their human contribution is “meaningful and more than de minimis,” the rules now state. An update to Grammy awards’ eligibility criteria states that “[o]nly human creators are eligible to be submitted for consideration,” and that “[a] work that contains no human authorship is not eligible in any Categories.”
DeepMind is using techniques from AlphaGo, DeepMind’s AI system that was the first to defeat a professional human player at the board game Go, to make a ChatGPT-rivaling chatbot called Gemini. If all goes according to plan, Gemini will have the ability to plan or solve problems as well as analyze text, DeepMind CEO Demis Hassabis told Wired’s Will Knight.
The well-funded AI startup took the wraps off the large language model powering its Pi conversational agent. The model is called Inflection-1 is of roughly GPT-3.5 size and capabilities, as measured in the computing power used to train them. According to the results they published, Inflection-1 indeed performs well on various measures, like middle- and high school-level exam tasks (think biology 101) and “common sense” benchmarks (things like “if Jack throws the ball on the roof, and Jill throws it back down, where is the ball?”). It mainly falls behind on coding, where GPT-3.5 beats it handily and, for comparison, GPT-4 smokes the competition; OpenAI’s biggest model is well known to have been a huge leap in quality there, so it’s no surprise.
Salesforce announced that it’s growing its Generative AI Fund from $250 million in size to $500 million. The Generative AI fund has already invested in several firms on the frontier of generative AI tech since launching in March. While far from the only fund investing primarily in generative AI, Salesforce aims to differentiate its tranche by prioritizing what it describes as “ethical” AI technologies.
GPU maker Nvidia was doing fine selling to gamers and cryptocurrency miners, but the AI industry put demand for its hardware into overdrive. The company has cleverly capitalized on this and the other day broke the symbolic (but intensely so) trillion-dollar market cap when its stock hit $413. They show no sign of slowing down, as they showed recently at Computex…
Among a dozen or two announcements at Computex in Taipei, Nvidia CEO Jensen Huang talked up the company’s Grace Hopper superchip for accelerated computing (their terminology) and demoed generative AI that it claimed could turn anyone into a developer.
Altman was recently advising the U.S. government on AI policy, though some saw this as letting the fox set the rules of the henhouse. The E.U.’s various rulemaking bodies are also looking for input and Altman has been doing a grand tour, warning simultaneously against excessive regulation and the dangers of unfettered AI. If these perspectives seem opposed to you… don’t worry, you’re not the only one.
We kind of spoiled this news for them when we published details of this fundraise and plan ahead of them, but Anthropic is now officially $450 million richer and hard at work on the successor to Claude and its other models. It’s clear the AI market is large enough that there’s room at the top for a few major providers — if they have the capital to get there.
Video social networking platform Tiktok is testing a new conversational AI that you can ask about whatever you want, including what you’re watching. The idea is instead of just searching for more “husky howling” videos, you could ask Tako “why do huskies howl so much?” and it will give a useful answer as well as point you towards more content to watch.
After investing hundreds of millions into OpenAI, Microsoft is determined to get its money’s worth. It’s already integrated GPT-4 into its Bing search platform, but now that Bing chat experience will be available — indeed, probably unavoidable — on every Windows 11 machine via an right-side bar across the OS.
Google is playing catch-up in the AI world, and although it is dedicating considerable resources to doing so, its strategy is still a little murky. Case in point: its I/O 2023 event was full of experimental features that may or may not ever make it to a broad audience. But they’re definitely doing a full court press to get back in the game.
Age of AI: Everything you need to know about artificial intelligence by Devin Coldewey originally published on TechCrunch
from TechCrunch
via
Click me for Details
