I haven't seen the paper, I highly recommend you go read it, because it's basically proofs
one and for all, that this is not really what you're still looking for, right?
You know, like everybody was talking how much the Nvidia price is dropped, and you know,
how much we're now really entering a different era, but I think James Allen basically
summed it up the best.
So James Allen, Executive Director of Linux Foundation, again, we didn't know in the open
source community, basically made this point, this is a golden age for low-level open source
AI engineering, and hearing it from him was especially useful because it's really reminded
me of something that I experienced myself.
Like how many of you guys recognize this email that got sent around in 1991, so like one
particular mailing list, you know, anybody in the audience recognizes that email at all,
and I have a few hands, I mean, I guess we're all farce's still showing up at Fuzznam.
This is line of store votes announcing Linux, when nobody thought that a group of individuals
let alone one person would basically compete with Microsoft, because like Microsoft and IBM at
that point, you know, yeah, not all of you remember, was it both to own it all?
Like that was in, that was the landscape of the computer industry at the time, and there
was like some micro systems on the French, but you know, that was it, and that email changed everything.
I was in this is everywhere.
If you compare it to the email, while essentially it wasn't an email that more like a GitHub
you know, you shoot that G-O-D-G-R-G-R got a positive, it's actually kind of amazingly
similar, right?
You know, several principles, you know, like, I'm going to do the thing mostly for fun,
it's simple enough, this is what true engineering looks like, and in fact, I will try
to kind of tie it all back to G-P-C, in fact, that's exactly what G-P-C proved.
Because if you read that paper, you get this real clear understanding that the view
for you, it come up with, you know, too many mobile, you know, rocket science level tricks
on the machine learning site, they really did just some clever, you know, very sophisticated
engineering, and that's what we're all here for, you know, to understand how computers can
basically execute these types of systems as well as possible.
So now that we're talking about it, and again, just as an introduction, what is it that
we've been talking about?
Because everybody says, yeah, you know, machine learning and deep learning, and like
people get confused, I actually really love this little book of deep learning, it's a
high professor who also happens to be like a really good engineer.
So download it, read it, it's about 100 pages long, you will understand what AI is much
better if you read it.
But I love this definition, that the AI is an application, it's basically a business
use case of its technology, right?
So when you say that I'm doing AI, I don't really have to talk about the application.
Engineering wise, we're talking about deep learning, which is an application of, you know,
machine learning, focused on learning from representation, and the way we do it to put it
in the engineering speed, we take this thing for the neural network.
It's sort of even some kind of an architecture, and that's slightly out of scope for us
engineers.
That's where the machine learning people, you know, come up with the next best architecture,
you know, it's renformers versus mom versus three versus that, but then we optimize the
heck out of it by running that thing on the computer.
And we used to run those things on computers, you know, through a variety of different networks,
mostly done by people in academia, and hence not really suitable for, you know, a quick, sort
of entroyant documentations that, again, I expect to see anything that runs on the computer.
It's very flexible, it allows you know, machine learning people to prototype extremely quickly,
you know, get to the results, you know, it's very valuable piece of software that they give
in us, but it's not really something that can be optimized, that is not in my case.
Although some people try it, so there's actually an interesting computer company in that
called H, that is making transformers, you know, be H, like, you know, hence the name, right
to the silicon.
So they're basically producing cases that are only every capable of running one architecture
in that is transformers.
And that is the way to optimize it.
Now, again, if you can't, if your model is not a transformer, you know, you all have to
flag.
But to me, this is an extreme case, because I and the guy who like to a general purpose computing,
in terms of general purpose computing, all of these frameworks kind of went through these
iterations.
So they used to be free history, because again, machine learning, you know, started way back
in the sixties, you know, a lot of the architecture is like a demo, you know, for example,
I still with us in the same way that they got introduced in the sixties.
Amazingly enough, the framework of the diverse MATLAB, because MATLAB is actually super
old, you know, not a lot of people know that.
And that was kind of a pie torture of the day, like, you know, your grandfather would go
to work at IBM and use MATLAB to basically do AI.
I mean, they would quote that way back then, but that's what happened.
And nothing really, not too many artifacts survive, although if you Google, if you're kind
of like Google, you know, PhD thesis, it's amazing how much, you know, what they developed
back then, it's still applicable today, but you have to translate it through a lot of historical
baggage.
Then, you know, what is interesting, I think, would basically have this revival of AI
or computers from actual computers starting at the bowler 2012, because 2012 was then,
there was this guy, you know, who realized that now he can actually run these types of
architectures on a computing unit called the GPU, because before it was, again, general
from his computers, you know, yeah, fast but not fast enough, all of a sudden he got
his hands on the bunch of MDGP's, and he's like, heck, this buddy of Maya, so, you know,
sound you will probably recognize Alex Net, a very, you know, seminal paper.
So two people wrote it, why was the machine learning engineer, and Alex, and Alex Net,
the other, he was more on engineering engineer, but basically figured out all of these details
of how to run it as fast as possible, or, you know, a pretty large number at the time
of GPUs.
And hands, you know, arguably, you basically have to put that as his main way of doing it,
because, like, he didn't even have its framework at the time, right?
If you go and look at the original source code, you know, he was just put it, like, he literally
was programming into the kernels.
It's all for actually, it's funny, because if you go to GitHub, there's a lot of
implementations of Alex Net, intensify the flow, and in PyTorch, and elements
complaining that it's slower than what Elia did back at the time.
Like, it's really easy, Elia is like, how can we actually, because they take their
original implementation, they want to run it side by side with the framework, and
like, the framework is always slower.
Like, what the hell, like, how can we get back to the original numbers?
Now, that was the time when we go all realized that we should probably arrive
on specialized hardware, and Google enters the race with basically designing
not a GPU, but a TPU, right, you know, tons of processing units.
Again, a PhD thesis behind it, now this time, Google's like, no way, we actually
have to give people a framework so that they can use this technology, because nobody
is smart enough to figure out how to write compute kernels for it.
So they basically come up with them, so flow and the rest is history, but in parallel,
a team at Google was also working on a next-generation architecture,
transformers, and all of those three things, you know, kind of compliant to create a
perfect storm, so transformers will go out of that effort.
Finally, if you look at 2015, open AI, it went back to India, took transformers architecture,
trained with on a whole bunch of data, by talking with this thing to do, and that
was what I call ML PhD era, we were experimenting with networks, you know, there was
not really a lot of sophisticated optimizations put into it, but that was fine, and that
is what people are still associated with AI today.
What I want you to pay attention to is this last slide on this slide, I call it an engineering
era, and that is basically if you look at the architecture of the networks, they kind
of fixed by now, right, it's kind of transformers plus class, you know, maybe it's
Mumbai, maybe it's that, but we as engineers don't really have to worry about it too much.
Make sure of experts is actually a great thing, because it allows you to partition your
own networks effectively in smaller branches, and we should really do a lot of it.
Now, what is really super interesting to me is that a whole bunch of next-generation frameworks
came out of this effort of kind of like looking at it and saying, how can we run inference
on this type of architectures and maybe something, you know, but I think a lot of I'm
actually the focus on inference, as quickly as possible, and these are the things that we
will be talking about through the day today.
GGML, ZML, tiny girl, there's a project that we haven't mind, that all of these frameworks
that are completely underpercated, and yeah, they are basically the basis of the deep sea type
of an event, because that is where the engineering is happening, and if you're a software
engineer like myself, if you're like, you know, tinkering with bits and bytes, this is where
you need to spend your time on.
Now, again, what are we going to run it on?
I actually have a prediction to make, so catch me here in, you know, five years, and, you know,
let's see that prediction depends out.
I don't think we'll be running it on CPUs, saw, not all.
I definitely don't think we'll be running it a lot on GPUs.
I think a transduer-like architecture with the rest five main course, you know, type one approach
would be an example of the kind of a thing for which these types of frameworks will be
optimized.
So, yeah, I give you, you know, at least in my view, the inference stack of the future.
You basically have a piece of silicon that co-evolves with the, one of these frameworks.
Again, could you channel tiny grant, GML, like, I don't care.
There's a bunch of things that you kind of have to do on top of it, because inferencing
in model is not just multiplying matrices.
There's a lot that comes before that.
And then, give your customers your login friends APIs, fine tuning APIs, enterprise APIs,
management APIs, and you sell it as a product.
So, if you're interested in developing anything like that, I think this is roughly the area
of the scope that, you know, you should be paying attention to.
And that is exactly why it is never on today.
You're so different from all of the AI conferences, I've been, you know, in the past few years.
And I hope the rest of the speakers will make the point I'm trying to make much more eloquently,
but I just want to do to know all of this.
Thank you so much.
Thank you, Roman.
Oh, switch out could.
Okay?
I'll show you.