Encoder Job Distribution Update
Hello my Encoding Frenz!
I'm Working late again — because such is life. Right?
You may have noticed today that your encoder was pretty lazy. Yes, it’s a little bit my fault… but I can explain.
What Was Wrong
The Encoding Gateway has a particular way of tracking which encoders are actually online. The last seen field is updated based on the last job your encoder finished.
In other words: the system only updated your encoder’s “last seen” timestamp while you were actively working on a job (via the 5-second ping). However, when your encoder was idle and simply polling for new work every minute, the gateway wasn’t updating this timestamp.
After 5 minutes of being idle, the gateway considered your encoder “offline” and stopped assigning jobs to it.
(muh bad!)
Result
Only the encoders that had recently finished jobs remained “visible” to the job distribution system. Everyone else became invisible — despite being fully online and ready to work.
How I Punched Murphy in the Face
I updated the system so that every time your encoder polls for work (every ~60 seconds), the gateway now updates your “last seen” timestamp. A heartbeat, the nerds among us will shout.
This keeps all idle encoders marked as active and eligible for job assignment, and everybody happy.
Why We’re Moving to Round-Robin?
Yes, I changed the job distribution logic — and bigly so, to quote a famous “leader.” (note the quotes)
We now use a simpler round-robin system that assigns jobs to the encoder with the fewest active jobs. This ensures:
Fair distribution – Work is spread evenly across all online encoders.
No race conditions – Jobs are pre-assigned by the gateway, eliminating the old “first come, first served” chaos (and infighting).
Better efficiency – Prevents a few fast encoders from hoarding all the work while others sit idle.
The old system had encoders racing to claim jobs, which led to duplicate assignments, wasted resources, and uneven distribution.
The Encoding Gateway now works as the single source of truth.
What You Need To Do: Git pull? Anyone? 🛠
Sorry, I know, I'm a scratched record at this point. But, I promise your encoder will automatically benefit from this tweak. It won't take you long either, and by now you know the drill all too well. I make you practice after all.
git pull, npm run build, and restart your encoder - easy peacy lemon squeezy?
Forgiveness?
Today was one of those days, but honestly today does not compare to how things were months ago. I do apologize for the short outages, and the inconvenience of having to update the code all the time. But these are growing pains, as the grandpa's say, and I think we are all trying to grow.
I hope these changes make the system fairer for everyone running an Encoding Node. After all, that is the one thing we all care about.
I’m very grateful to you all, and again, I apologize for the murphycation of the day. Now, I sleep.
zzzz 🛌
- MenO
Leave Encoder Job Distribution Update to:
Read more #hive posts
Best Posts From MenO
We have not curated any of meno's posts yet. But you can encourage our curation team to review posts by visiting them regularly and by referring other readers. Because we give priority to frequently read content.
More Posts From MenO
- I Taught My AI to Talk Behind My Back
- I Love Philosophy. I Just Don't Think I'll Ever Become a Phil Bro.
- Meniski Just Got a Job
- Where Choice Ends and Belief Begins
- Apple Said Yes. I Wasn't Expecting That.
- The Encoding Gateway Has Officially Retired
- Today Was a Good Day for Snapie
- Curators Need a Game Too
- The Man Who Almost Killed Bitcoin
- Are We Printing Hive Into the Future?
- Open Source, Hive, and the Art of Buying Bread
- Everywhere I look, red.
- The War Between Simplicity and Features
- The Case for an AI Intern at ThreeSpeak
- The Fun Deficit
- The Value of Uncomfortable Conversations
- Ideas, Ideas and Ideas
- A Few Hiccups, A Few Upgrades
- But Boy, Does My Jaw Hurt
- Who Are the Content Consumers?