ML@ABEJA

Microsoft’s Orca 2: A Leap Forward in AI Reasoning

Introduction

Beyond Limits: Unifying Strategies to Amplify Transformer Models

Enriching Computational Workspaces…

Flexible batchsizing for dataset with very diverse sequence lengths

Using GPU compute more efficiently (for transformers)

Dynamic batch-sizing for dataset with vastly diverse sequence lengths

Extending AutoModel class from Hugging Face

A hacky way to keep the code structure consistent

Setting up a FastAPI server with GPU

Using serve from ray

With an aim to make my machine learning model usable for public I could successfully hosting with CPU support only using the the following code snippet.

from fastapi import FastAPI, File…