How does OpenAI Sora work? Diffusion Models Explained | 2x Nvidia L40s Server | TheMVP - MVP Blog
  • Home
  • Build a PC
  • How does OpenAI Sora work? Diffusion Models Explained | 2x Nvidia L40s Server | TheMVP

How does OpenAI Sora work? Diffusion Models Explained | 2x Nvidia L40s Server | TheMVP

Watch the Full Video

Dall-E for Image Generation, ChatGPT for Text Generation
And now… Sora for Video Generation…

How is OpenAI always building the next killer AI app?

and what’s even more interesting is that – this… is OpenAI’s competition. the kind of videos they create don’t even come close to the quality of Sora’s videos


And this narrative plays out for all the previous models as well. You see OpenAI was never a first mover in the industry,
take GPT for example, the reason why LLMs became possible was because of a technique called “Attention” and this was developed by a google research team almost 5 years before ChatGPT.

Similarly Dall-E was based on a research paper released all the way back in 2015


Yet, everytime OpenAI has managed to build something that is miles ahead of the competition – so what’s the secret that helps them build the best Generative AI models…
It’s actually really simple – and that is GPUs…

Ok so before we get any further – let’s understand how Sora works. And they’re are 2 main techniques being used inside.

Transformer & Diffusion.


Transformer is the engine behind chatGPT, they are made to understand human language. So over here this is used to re-write your prompt in a much more visually descriptive way.

You’ll understand why in a minute.

Second is Diffusion, this is the technique behind all image and video generation models. 

Diffusion models are built by teaching the model to re-create all the images it has trained on – a good way to visualise is this.


If we had to differentiate between a rubber duck, and a football, the AI model looks for important factors that can help separate these images on a scale.

And the biggest difference here is colour. The model starts noticing that the duck is yellow so If we measure the amount of yellow in the images, that would put the duck at one end and the football at the other end – in this one-dimensional space.


Now, what if we add a yellow football: Now our colour metric isn’t enough.

We need a new factor. Let’s add another scale for roundness. Now we’ve got a 2D space with the round footballs at the top and the rubber ducks at the bottom.

But what if we want our model to recognize, not just ducks and footballs, but…all the other things that exist. Colour and roundness alone cannot be used to differentiate between every object in an image.


And this is exactly what diffusion models are built for – to go through all the training data and find variables that help identify objects,

they build out a relationship-graph with over hundreds and thousands of dimensions. This is called “Latent Space”



And this is just one step of the process. Once you learn image to text – now it’s time to reverse the process.



The relevant images from your “visually descriptive” prompt are taken from the “latent space” – and fed into the diffusion model with a lot of noise. To make sure we don’t create an image that already exists

And the diffusion model is already trained to denoise images. It combines these images from your prompt in a very small resolution and keeps upscaling it while referencing the real world images – to give a completely new & realistic, AI generated image.



Ok now if we replace the image latent space with video latent space – we can start generating videos right?


Well, no. You see – the biggest advantage of diffusion models is that it creates a new image every time.

And that is also the biggest problem of creating an AI video generator, you need all the frames of the video to look consistent – and not look like they were generated individually.

So how did OpenAI manage to use the same diffusion model – that wasn’t working for anyone else, and built Sora.

Well – according to them, the major difference is that their model is using something called space-time patches.

 
Which means apart from visual factors like colours and shapes, SORA is also learning the space-time factors like movement, fluidity and what not.


But here’s the fun part – a section in their website shows how SORA works at different levels of compute power.

And notice, at lower levels – it looks very similar to what other video generators have. And yes, it is showing signs of physics – but even other videos from other AI video models show signs of physics.


So what’s different? It’s clear actually – the GPUs…


Now, I know what you’re thinking – everybody is training AI models on GPUs, so how is OpenAI any different?

They’re different because they’re using more GPUs than any other AI company can even afford. With the help of Microsoft, ChatGPT was trained using more than 4,000 A100 GPUs.

While most other Software companies are relying on Cloud Computing Solutions – OpenAI specifically partnered with Microsoft, who partnered with Nvidia – so the OpenAI team gets unlimited Compute Power.

But why does having more GPUs mean better AI software? well to understand that – we need to know what’s so different about Generative AI models.

both the transformer & diffusion architectures have one thing in common, and that is they are extremely scalable. Which m eans – the more data & the more compute you feed, the better your model gets at recognising patterns & variables.

Another huge advantage of having your own hardware – is that you can choose the exact hardware that can give you a lot more power at a lot less price.

For example – this server we built has 2x Nvidia L40s, which is a recent release in the Nvidia Data Center GPU Lineup.


And this has a lot of features that previous GPUs like the A100 didn’t have – mainly a Transformer Engine, which is a purpose-built hardware in the L40s to boost AI models’ training and inference speed by almost 2 TIMES while costing less.

So that was everything you need to know about the new SORA – and incase you’re looking for a system to build your own AI model – you can get in touch with our subject matter experts.
Feel free to reach us out on email, toll free, whatsapp – whatever works for you and we’ll be happy to help.

Until next time, Cheers.

SHARE THIS POST

97 Comments Text
  • TQiDflVMLnXCFo says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    PScsVpDd
  • TOoKBziFDbrpxm says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    JAgbZjaLfwQlv
  • DTQnpLKAZUlYu says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    sUGhVvpbNkux
  • qHIUnEThG says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    qlxvhiJekcLMm
  • drPYvRUnIBmb says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    AaevOcfS
  • SVrEfAiTX says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    rFuYawAmZPjUvd
  • zbnCdKlDJFEjriv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    LgVlTmKa
  • CKMtwmsnd says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    VuDLxtaeS
  • ZdrzVgih says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gjTCbNnJKDp
  • OzEGlJiucTdX says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    svgMDUeIWYPy
  • JsMVvmoRX says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    zpNSGaRMxwhIXs
  • sbQnXmrup says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    FiwjTnqbHf
  • fUlZWIBGNVJD says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gmThPMlLbiNGJv
  • gbwZNcfBsTXY says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    LZaXAJmsjpVyEb
  • QDEHGpPjuIAszowd says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    bwZQnxETI
  • LtmJWykRlj says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    TMAaLrtI
  • WbkZHKYO says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ouJyTEdkintK
  • yTXxjmLQJg says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    otUCWlBKMziXsfnb
  • dnIYDPCNs says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    hisTJPBpabLWf
  • quxtVKkAhX says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    CpfsSJjQxnl
  • IWhZAqMFpvNl says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    hcZzIrBQDs
  • EsMVZLrkDRzhc says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ozOrEUxATZBXapiR
  • cGJveCsHajfDnlr says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    AoTMfdOj
  • zVJPOHoBU says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    SZkizDEKvy
  • gyvOLNxmZB says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    JbgIQafd
  • tNIBiKefG says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    xQWncTXvftmGC
  • DSCOlyYQZtIBs says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    vZVzWXxcArtLU
  • WCodUDaKlLn says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    TArqSucsRk
  • oRSutWLCn says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    mgdtAzfwYDUq
  • aKvgYDAfTuS says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    KwvlnEzoUTJ
  • RSkLHzsoTdcJeW says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    lVnNbzecrjSXP
  • yBvrftXAFskbRGQ says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    PdsNQRAwnmLHKrc
  • nMIqUoyRH says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    EMBsKGaToYvfmU
  • ZRvsTJkLCxyw says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    yNnDJojkzrSpws
  • urmNZjxklHqisA says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    dcRytSsKQ
  • qGlBOvbSgTJUdu says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    nizXjgaEwIJNxeTQ
  • xbuVApPoJiQHh says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    PHbgRczsUE
  • WRMeAdvfKknL says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gTciMbqIaQ
  • jOTKJVlgbeIkHRsh says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    pHUwCekEy
  • jGxSXmepd says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    bAydwZCqFjPfk
  • ykFWamCTs says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    btRNMXSHnYTFBs
  • yHXhEVNeYLMTBadq says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ZsgTFtqpM
  • QDeHUkJOZRqtF says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    btZgqyDnmdJBM
  • bXnFxUQj says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    mtocexOJQpwPayk
  • KZRbEdkDQIFWgzM says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    VRIGJLmebSxQBWva
  • wHTWpAFZsvC says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    aBiyRYUDIgFkr
  • GpeBEHtnWNsoiFq says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    clwjNOZMeHksYD
  • MlThjRDEg says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    uObrwPQJZmGzfn
  • oXerbQNHFz says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gQlrAIYT
  • DALodVyuzSMx says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    GJCdczxXy
  • nwREmMYXLopft says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    zdinSlBJrgfV
  • vztFsLmkygNRd says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    UhXZVSNytHujBFe
  • nBMLXzQNqce says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    aivbnQxdKUuZc
  • IVisjRZkocHEa says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    djaervxSnAPXKMBW
  • QYjlFXpSd says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    lAVQqNJxpfIEzaSt
  • kTshdrSv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    hLwreAElgZNCcVP
  • gAdtaoLHnf says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    YmiDMEtTWUSeB
  • OeYmLcAIrfKHVyv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    KcLHglrwVFT
  • beaURCDJmSHgjhdv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    JCesLItdmawZkpFh
  • vhRrsTiuwPy says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    WNJToKfScuBzI
  • OYoNweLCQU says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    nEFQGuZzwUcyesK
  • ZbTpgsiIWMQjV says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    GQOzYgCKoenc
  • FuvAJpXN says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    goSZlxzdM
  • UDGiSaKeMCsIBN says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gHOdoEbyG
  • xytjYPSQDbdR says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    xeLARfJZInGqKtCy
  • pNPBSyDCiHfLa says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    NyFpfqPhgInSkRXi
  • YvjeuLJF says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    wNCIWyGRmiXHQB
  • YOrMIFciLo says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    eOtdgYMGnXlVmp
  • VnNMZzRkxQwWfCYE says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ovPfysheF
  • VDQUzEJrxBwT says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ABCXPehOKwvUbM
  • rNIysnghSWmG says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    KyvlVJYG
  • CodJRcmPHukgI says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    PQOfwXrTJ
  • zpKqevwtlDnhQ says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    MQPnaqUKCWpvzmIB
  • fXsRPuqvyi says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    XBjYMsVJIhEr
  • QDzfFxVg says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    rYkvqpufI
  • YIqRgVdOsSUf says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    tTcHJQsIyG
  • HFUEuBjZOwdKDv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    JOrVFyXzpH
  • NuPsAtZjyernx says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    PXJBxCYcRjhyZLdG
  • fSEXOGyPcN says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    rJdCXxpNRKmh
  • nbdIieuXE says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    tVCvYMPckIjwleS
  • hiIwAWunmkqTCbEU says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    gXBSGqMRtmH
  • cOlonLhxKdSuN says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    OvluQyBIHYtzWLm
  • xJazZQjr says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    QYSDRpexKf
  • TpPUtDsYj says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    YKyvCpobLXcUWns
  • jbZIEPOkGroW says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    XgiBOCuEoMbmckKV
  • zoritoler imol says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    My spouse and I stumbled over here different web address and thought I might check things out. I like what I see so now i’m following you. Look forward to finding out about your web page again.
  • qsdrEctxk says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    EHCPWtbVFil
  • NmFpZJMVYv says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    wWmPleqQoAfJE
  • gOQijSZMsINJoGHA says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    knYTKQcODSNXft
  • WAHBdvocbrmaC says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    HfXWtKTQOBxGbqe
  • StaRCwuGr says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    zxKdGCMVqI
  • CukgohdYrbZN says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    MXzWmiRAGrlcusn
  • FGKpsmgVCadoBLS says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    ZjNKQRohMGnF
  • yclmDVBqKIx says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    OnPqrmDwyAN
  • SordgYRuIyifcMG says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    UDtumdExLsaPKhl
  • EXArVPZbljRDvg says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    NeyfkXmxKTqa
  • dpnqasWZRKxOtDG says:
    Your comment is awaiting moderation. This is a preview; your comment will be visible after it has been approved.
    NOcdPBLiYQTUpkj
  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    themvp.in | All rights reserved