<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ojas Shukla on Medium]]></title>
        <description><![CDATA[Stories by Ojas Shukla on Medium]]></description>
        <link>https://medium.com/@ojasshukla01?source=rss-7ad81bc45931------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*6RyGkZ7AeD-UzOZ1</url>
            <title>Stories by Ojas Shukla on Medium</title>
            <link>https://medium.com/@ojasshukla01?source=rss-7ad81bc45931------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 28 May 2026 03:13:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ojasshukla01/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[The Architecture of Curiosity: Crafting a Data Engineering Portfolio from Invisible Work]]></title>
            <link>https://medium.com/@ojasshukla01/the-architecture-of-curiosity-crafting-a-data-engineering-portfolio-from-invisible-work-d58ff32e3ee9?source=rss-7ad81bc45931------2</link>
            <guid isPermaLink="false">https://medium.com/p/d58ff32e3ee9</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-engineer]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-architecture]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Ojas Shukla]]></dc:creator>
            <pubDate>Sat, 25 Oct 2025 11:24:26 GMT</pubDate>
            <atom:updated>2026-01-30T10:03:57.685Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>In the midnight silence, a lone programmer conjures an entire world out of code.</blockquote><p>The screen’s glow paints a determined face, lines of logic dancing like musical notes across dark terminals. I often liken this feeling to standing in the rafters of an unfinished cathedral, seeing the grand structure in my mind before a single brick is visible to others. This is how my journey began, inside the mind of a systems architect, where invisible engineering feats yearned to take shape in the daylight. I wasn’t just coding a personal website; I was trying to externalise an inner universe, to make others feel the <em>architecture of curiosity</em> that drives my work.</p><h3>Making Invisible Engineering Visible</h3><p>Engineering, especially data engineering, often happens behind closed doors (or rather, behind API endpoints and cron jobs). Stakeholders see the polished dashboard or the snappy application, but not the intricate pipelines, optimised SQL, and streaming workflows under the hood. My motivation for a portfolio was simple: make the invisible visible. I wanted to peel back the layers and showcase the creative engineering buried in those nightly ETL scripts and real-time data streams. Much like developing a photograph with special post-processing to reveal hidden detail, a well-crafted portfolio can expose the beauty in engineering work that usually goes unnoticed. As one developer noted, a personal site is an online representation of you and should demonstrate best practices, a sentiment I took to heart while building mine.</p><p>Instead of a static resume or a bullet list of skills, I envisioned an interactive exhibit. Every project on my site would be a story, every metric a number with rhythm rather than a dull statistic. For example, I built a small web app to pull live data from the Steam API (because if I’m showing off engineering, why not also show off some gaming stats?). This turned my profile into a living tableau where database queries and API calls perform in real-time, right on a recruiting manager’s browser. The goal: let visitors not just read about my work, but experience it.</p><h3>Designing the Stack: From Next.js to DuckDB</h3><p>To bring this vision to life, I needed a solid yet dynamic architecture, something reliable enough to score well on performance, but flexible enough to support interactive data storytelling. In true engineering fashion, I sketched a blueprint of the tech stack before writing a single line of code. Here were the high-level choices and why I made them:</p><p>· <strong>Next.js 14 - </strong>The foundation of the front-end. Next.js offered server-side rendering and the new App Router features, ensuring my site could be static, fast, yet dynamic when needed. (Using Next.js was a no-brainer for a modern personal site, combining the SEO benefits of static generation with the flexibility of API routes for dynamic content.)</p><p>· <strong>TypeScript - </strong>Every architect needs precision in their measurements. TypeScript brought type safety to my codebase, reducing bugs and making the development process more predictable. In a project showcasing engineering best practices, having robust, self-documenting code was a must.</p><p>· <strong>Tailwind CSS - </strong>For design, I reached for Tailwind. Its utility-first approach let me rapidly prototype and refine the site’s look without context-switching out of HTML. Styling became a joy instead of a chore (a far cry from my initial scepticism about Tailwind, which has since become an indispensable tool). The result was a clean, modern interface that doesn’t scream “template,” because I tailored every component to my content.</p><p>· <strong>Streamlit - </strong>Data engineers love data apps, so why not embed one in the portfolio? I used Streamlit to create a mini interactive analytics page within my site. It’s like a sandbox where one of my projects runs live, think of a mini-dashboard or a data viz demo that readers can play with. Streamlit handled the heavy lifting for charts and widgets, and I integrated it seamlessly via an iframe, making it feel native to the Next.js app.</p><p>· <strong>FastAPI - </strong>Behind the scenes, I stood up a FastAPI service to act as a bespoke backend for my portfolio. This service powers things like a contact form API and provides data endpoints for the Streamlit app and other interactive elements. By designing a lightweight REST API, I ensured that even if the front-end is static, it could fetch fresh data (for example, pulling the latest stats from that Steam API integration on demand). FastAPI’s speed and Pythonic simplicity made it an ideal choice for a sidecar service delivering JSON to my front-end with minimal latency.</p><p>· <strong>DuckDB - </strong>Data engineers have a knack for squeezing SQL engines into surprising places, and I couldn’t resist using DuckDB. This embeddable analytics database gave my portfolio a way to run analytical queries on the fly without spinning up a cloud warehouse. For instance, one project panel lets users filter and aggregate a dataset in real-time. DuckDB executes those queries swiftly in-process, showcasing “analytics at the edge”. It’s a nod to my day job skills, presented in a way a hiring manager can actually interact with.</p><p>· <strong>Steam API - </strong>Finally, the fun part. By pulling data from the Steam Web API, I turned a personal interest (gaming and esports) into a technical showcase. One section of my portfolio fetches live data about a popular game (Dota 2 match stats, to be exact) and visualises it. This isn’t just for flair; it demonstrates how I handle external APIs, real-time data updates, and data visualisation. Plus, it’s a conversation starter, a bit of my personality shining through the tech stack.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hw64vOiBY9JGaxiAjl9vhw.png" /><figcaption>Portfolio across various screens. Designed by Gemini</figcaption></figure><p>Each of these decisions was made to support the core mission: make the portfolio an engaging systems design in itself. I wanted someone who visits to not only read about cloud pipelines and data lakes, but to sense the design mindset in how the site is built - a <em>full-stack data narrative</em>.</p><h3>Challenges on the Journey (and Solutions)</h3><p>Building this portfolio was not without hurdles. I faced three main challenges, each a balancing act between extremes:</p><p>· <strong>Design vs. Clarity:</strong> As an engineer, I’m more comfortable in VS Code than in Figma. The challenge was creating a visually appealing site that still felt <em>authentic</em> to my style. I opted for a minimalist dark theme, accentuated with purposeful highlights (inspired by the idea that good design is as much about what you omit as what you include). Whenever I drifted into overly ornate UI territory, I remembered the mantra:</p><blockquote>“Design is not about what you add, but about what you remove.”</blockquote><p>The result is a clean interface where content is king. Code snippets, architecture diagrams, and data visuals stand out without needless clutter. And yes, I did indulge in a bit of creative CSS, after all, a dash of animation or a tasteful gradient can convey that <em>spark</em> of creativity in engineering.</p><p>· <strong>Performance vs. Interactivity:</strong> With all the embedded apps and live data, I worried the site might turn into a slow, heavy beast. This pushed me to optimise relentlessly. I used Next.js dynamic imports to load heavy components (like the Streamlit iframe) only when needed, and leveraged partial pre-rendering so initial loads stayed snappy. Images are optimised via Next’s built-in image component. I also took advantage of Next.js 14’s performance enhancements like the Turbopack bundler for faster builds and snappier hot reloads during development.</p><p>· <strong>The payoff:</strong> My portfolio consistently scores in the mid-to-high 90s on Lighthouse performance audits, even with interactive elements running. (One portfolio project even achieved a 95+ Lighthouse score despite complex animations, proving that speed and creativity can coexist.) To keep the site feeling alive without sacrificing speed, I implemented WebSocket updates for a couple of real-time stats, ensuring that new data flows in smoothly in the background. It’s like tuning an engine, finding the right balance of power and efficiency.</p><p>· <strong>Narrative vs. Brevity:</strong> Perhaps the trickiest challenge was weaving a narrative without overwhelming readers. It’s tempting to catalogue every project, every certificate, every tech I’ve touched. But attention is scarce. I decided to frame the portfolio as a journey - a story with a beginning (how I got here), a middle (what I’ve built), and an open end (what I’m exploring next). Each project write-up on the site reads like a case study vignette, focusing on the “why” and “how” rather than just the “what.” I also used analogies (for example, comparing a data pipeline to a brewery process in one project description) so that even non-data folks can appreciate the essence without wading through jargon. This narrative approach keeps readers engaged and lets my passion shine through in a relatable way. It turns out that storytelling isn’t just for novelists; engineers can harness it too, to great effect.</p><blockquote>A digital painting of a figure standing atop a complex network, surveying the landscape.</blockquote><p>This image captures how I often felt during the build: equal parts challenger and observer of the world I was creating. Every challenge overcome was like reaching a new vantage point, allowing me to see further and refine the journey ahead.</p><h3>Performance Outcomes: Speed, Accessibility, and Interactivity</h3><p>In the end, all those design and engineering decisions paid off in the form of tangible results. I didn’t just want the site to be pretty or clever. I wanted it to perform like a professional-grade application. Here are some highlights of how the portfolio scores:</p><p>· <strong>Blazing Fast Loads:</strong> Thanks to Next.js and careful optimisation, the site’s pages load in a snap. On Lighthouse audits, it scores above 95 for performance, meaning it loads faster than the vast majority of sites on both desktop and mobile. (Google’s Lighthouse scale rates 90+ as excellent, and I was thrilled to consistently hit that sweet spot.) The combination of static generation and selective hydration of interactive components gives the best of both worlds: instant content and engaging features.</p><p>· <strong>Accessibility 100:</strong> I paid attention to ARIA labels, colour contrast, and semantic HTML to ensure the site is usable for everyone. The effort resulted in a perfect Lighthouse Accessibility score (100). Even with custom graphics and dynamic content, I adhered to WCAG 2.1 standards so that screen readers and keyboard navigation work flawlessly. After all, <em>an engineering marvel isn’t very marvellous if people can’t use it</em>. Inclusive design was not an afterthought but a core part of the build.</p><p>· <strong>Real-Time Interactivity:</strong> The portfolio isn’t a static museum; it’s more like a science centre with live demos. The Steam API-powered section updates on the fly; you can see current data pulled in as of this minute. My Streamlit data app embeds quickly (thanks to caching and lazy loading) and allows users to play with data filters without leaving the page. Despite these dynamic features, the site remains stable and smooth. No janky animations, no freezing, just seamless interaction. The tech under the hood (FastAPI, DuckDB, etc.) works quietly to deliver a real-time experience in a static site’s clothing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*b4sWu8rm4WoEZEjUy5dsNA.png" /><figcaption>Stack Design. Designed by Gemini</figcaption></figure><p>One pleasant surprise: after deploying, I ran a full Lighthouse test and realised I even nailed a 100 in Best Practices and SEO categories, alongside the high performance and accessibility. It felt like getting an A+ on a report card. More importantly, it means future readers get a fast, smooth, and engaging visit, which keeps them exploring my content rather than bouncing off due to a slow load or broken link.</p><h3>Reflections &amp; Lessons Learned</h3><p>Building this portfolio was an <strong>education</strong> in itself. It forced me to step back from day-to-day coding and think like a product designer, a storyteller, and a performance engineer all at once. Here are a few takeaways from this journey that might help others build technical portfolios:</p><p>· <strong>Storytelling is a Superpower:</strong> Don’t just list what you’ve done, craft a narrative around it. A cohesive story (with you as the protagonist in your own tech adventure) will stick in the reader’s mind. It gives context to your skills and makes your portfolio more than a glorified resume. I found that framing projects as case studies or chapters made my work memorable and meaningful rather than just impressive bullet points.</p><p>· <strong>Show, Don’t Tell (Whenever Possible):</strong> If you can turn a static description into a live demo, do it. It’s one thing to claim you can build real-time data pipelines, but it’s far more impactful to let someone click a button on your site and see data update in real time. Even small interactive elements or visualisations can speak louder than text. Plus, building these mini demos will sharpen your skills - it’s a win-win.</p><p>· <strong>Performance and Polish Matter:</strong> Treat your portfolio as you would a production app. Optimise images, minify code, and handle errors gracefully. A snappy, error-free site reflects an attention to detail that employers love to see. As an analogy, imagine your portfolio is a sports car you’re taking to a show; you want it not only to look sleek but also to have an engine that purrs. Every extra millisecond or broken link is like a dent in the hood. Polish it out.</p><p>· <strong>Embrace Your Personality:</strong> Finally, let your shine through. Whether it’s weaving in your hobbies (hello, gaming stats!), using a bit of humour, or adding a personal blog section with your musings, this is what makes your portfolio unique. In my case, I blended my love for data with a love for storytelling and a dash of geek culture. It humanises the experience for the reader. Remember, people hire people, not just skill lists.</p><p>Throughout this project, I often reminded myself why I started: to celebrate the craft of engineering. By making the invisible work visible, I not only communicated my abilities to others, but I also gained a deeper appreciation for the <strong>art in the science</strong> of what we do. In the process of building an “architecture of curiosity” for others to explore, I ended up exploring new facets of my own creativity as an engineer.</p><h3>Closing Thoughts</h3><p>Building this portfolio wasn’t just a coding exercise; it was a journey of design thinking, storytelling, and rediscovering <em>why</em> I love engineering in the first place.</p><p>From conceptual sketches to the final deployment, every page taught me something new, about performance, aesthetics, and the art of balancing logic with emotion. It’s one thing to build data systems that work; it’s another to build experiences that <em>speak</em>.</p><p>This project helped me bridge that gap between function and feeling, between code and narrative.</p><p>If you’re curious about how data engineering can be equal parts creativity and precision, I’d love for you to explore my work, browse through the projects, and maybe find a little inspiration for your own journey.</p><p><strong>Dive into the full experience:</strong> <a href="https://portfolio-ojas-shuklas-projects-7dc8ad06.vercel.app/">https://ojas-de-portfolio.vercel.app</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d58ff32e3ee9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>