<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Edge-to-Cloud Data Pipeline at zero cost]]></title><description><![CDATA[Edge-to-Cloud Data Pipeline at zero cost]]></description><link>https://fakruls-data-journey.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 12 Sep 2026 01:41:52 GMT</lastBuildDate><atom:link href="https://fakruls-data-journey.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Architecting a $0 Edge-to-Cloud Data Pipeline for a Poultry Farm]]></title><description><![CDATA[Introduction: The Restart
After a tough loss with our previous quail farm, due to Gumboro disease (a highly contagious viral infection), my parents and younger brother are back at it.
The R&D Project: 500 Sonali chickens.
While I sit at my desk, they...]]></description><link>https://fakruls-data-journey.hashnode.dev/a-zero-dollar-edge-to-cloud-data-pipeline-for-a-poultry-farm</link><guid isPermaLink="true">https://fakruls-data-journey.hashnode.dev/a-zero-dollar-edge-to-cloud-data-pipeline-for-a-poultry-farm</guid><category><![CDATA[agritech]]></category><category><![CDATA[Software Engineering]]></category><category><![CDATA[iot]]></category><category><![CDATA[Build In Public]]></category><category><![CDATA[Cloud Computing]]></category><category><![CDATA[edgecomputing]]></category><category><![CDATA[ Edge AI]]></category><category><![CDATA[supabase]]></category><category><![CDATA[streamlit]]></category><category><![CDATA[data-engineering]]></category><dc:creator><![CDATA[Md Fakrul Islam Taraque]]></dc:creator><pubDate>Sun, 15 Feb 2026 16:17:30 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1771170515336/01b755bd-4a77-4df6-8b3c-b0a05fe65a66.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h3 id="heading-introduction-the-restart">Introduction: The Restart</h3>
<p>After a tough loss with our previous quail farm, due to <strong>Gumboro disease</strong> (a highly contagious viral infection), my parents and younger brother are back at it.</p>
<p><strong>The R&amp;D Project: 500 Sonali chickens.</strong></p>
<p>While I sit at my desk, they are doing the real heavy lifting.</p>
<p>The physical reality:</p>
<ul>
<li><p>Smelling the air to detect ammonia levels 👃</p>
</li>
<li><p>Manually checking rice husk humidity</p>
</li>
<li><p>Reading chick "clustering" patterns to see if they’re cold</p>
</li>
<li><p>Inspecting poop for gut health</p>
</li>
<li><p>Mixing medicine and warm water manually for each refill</p>
</li>
<li><p>Treating rice husk bedding for hygiene maintenance</p>
</li>
</ul>
<p>I can’t help with the physical labor from a distance.</p>
<p>But as a Software Engineer, I wanted to remove one worry from their workload: <strong>Environmental monitoring.</strong></p>
<h3 id="heading-the-constraint-0-budget">The Constraint: $0 Budget</h3>
<p>I immediately went to spin up a project on Google Cloud Platform (GCP). I hit a wall instantly: a mandatory <strong>$50 prepayment</strong> just to verify the account.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1771170740790/3e038229-af6b-4940-a797-93410be12a50.png" alt class="image--center mx-auto" /></p>
<p>For a fragile family business in recovery mode? That was a no-go.</p>
<p>I needed an enterprise-grade logic on a hobbyist budget. I needed to build a <strong>Full Data Lifecycle</strong> (Ingestion → Storage → Visualization) for <strong>0 BDT ($0)</strong>.</p>
<h3 id="heading-the-architecture-the-broke-engineer-stack">The Architecture: The "Broke Engineer" Stack</h3>
<p>I mapped the standard Data Engineering Lifecycle to free-tier tools:</p>
<ol>
<li><p><strong>Generation:</strong> Python Scripts (Mocking DHT22/MQ135 sensors).</p>
</li>
<li><p><strong>Transport:</strong> <strong>HiveMQ</strong> (Free MQTT Broker).</p>
</li>
<li><p><strong>Ingestion:</strong> <strong>Render</strong> (Hosting the Python subscriber script).</p>
</li>
<li><p><strong>Storage:</strong> <strong>Supabase</strong> (PostgreSQL for time-series data).</p>
</li>
<li><p><strong>Serving:</strong> <strong>Streamlit</strong> (Data visualization).</p>
</li>
</ol>
<h3 id="heading-step-1-data-generation-the-sensors">Step 1: Data Generation (The Sensors)</h3>
<p>Since the physical sensors (DHT22 and MQ135) are currently shipping, I wrote a Python script to simulate the edge devices.</p>
<pre><code class="lang-plaintext">{
    'device_id': 'esp32-mock-01', 
    'timestamp': 1771171288.1688237, 
    'temperature': 28.66, 
    'humidity': 62.92, 
    'ammonia': 1.09, 
    'status': 'active'
}
</code></pre>
<h3 id="heading-step-2-the-storage-layer-supabase">Step 2: The Storage Layer (Supabase)</h3>
<p>I chose Supabase because it gives me a full PostgreSQL database on the free tier. I created a simple table to store time-series data:</p>
<pre><code class="lang-plaintext">create table if not exists brooder_telemetry (
  id bigint generated always as identity primary key,
  device_id text not null,
  timestamp timestamptz not null default now(),
  temperature float,
  humidity float,
  ammonia_ppm float,

  -- Store raw JSON payload for debugging or schema evolution transparency
  raw_data jsonb,

  -- Metadata for auditing
  created_at timestamptz default now()
);
</code></pre>
<h3 id="heading-step-3-the-challenge-render-amp-the-sleep-problem">Step 3: The Challenge (Render &amp; The "Sleep" Problem)</h3>
<p>The biggest engineering hurdle wasn't the code—it was the infrastructure limitations.</p>
<p>I deployed my MQTT subscriber script to <a target="_blank" href="http://Render.com"><strong>Render.com</strong></a>. It worked perfectly... for 15 minutes.<br />Render's free tier spins down web services if it detects no inbound HTTP traffic for 15 minutes. Since my script was a background worker listening to MQTT (not HTTP), Render thought it was idle and killed it.</p>
<p>(Docs: <a target="_blank" href="https://render.com/docs/free#spinning-down-on-idle">https://render.com/docs/free#spinning-down-on-idle )</a></p>
<p><strong>The Zero-Cost Solution:</strong> <a target="_blank" href="http://Cron-job.org"><strong>Cron-job.org</strong></a><br />I didn't want to upgrade to a paid plan yet. So, I added a dummy HTTP endpoint to my Python script and set up <a target="_blank" href="http://Cron-job.org"><strong>Cron-job.org</strong></a> to ping that endpoint every 10 minutes.</p>
<p>It tricks the server into staying awake—keeping the pipeline alive 24/7 without a credit card.</p>
<h3 id="heading-step-4-visualization-streamlit">Step 4: Visualization (Streamlit)</h3>
<p>For the interface, I used Streamlit. It connects directly to Supabase via SQL and renders the charts. It’s lightweight, Python-native, and free to host.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1771171700670/7ca84437-33fd-47f7-82d0-2a4207e7a46c.png" alt class="image--center mx-auto" /></p>
<h3 id="heading-the-result-amp-roadmap">🚀 The Result &amp; Roadmap</h3>
<p>We now have a live V0 dashboard monitoring the simulated environment.</p>
<p><strong>Project Roadmap:</strong></p>
<ul>
<li><p><strong>V1 (Automation):</strong> The next step is connecting the relay switches to control exhaust fans based on the temperature data.</p>
</li>
<li><p><strong>V2 (Edge AI):</strong> I plan to use <strong>OpenCV</strong> to analyze the "clustering patterns" of the chicks. If they huddle together, it means they are cold—a visual data point that sensors might miss. For now, it’s a simulation.</p>
</li>
</ul>
<h3 id="heading-conclusion">Conclusion</h3>
<p>This isn't <strong>Smart Farming</strong> yet. My code doesn't mix feed or change the bedding. But it’s a solid V0, and I'm hoping my code becomes half as resilient as my parents.</p>
<p>But you can see the live V0 dashboard here:</p>
<p>👉 <a target="_blank" href="https://agri-stack-dashboard.streamlit.app/">https://agri-stack-dashboard.streamlit.app/</a></p>
<p>Time to move from simulated data to real Edge IoT. 🚀</p>
<p>#IoT #AgroTech #SoftwareEngineering #BuildInPublic #CloudComputing #EdgeComputing #Supabase #Streamlit</p>
]]></content:encoded></item></channel></rss>