How it works
How technical AI annotation work works
AI companies need people who can read code to judge what their models produce. Here is what the work involves, what a task looks like and how applying works.
Why developers are needed
Models that write code improve through feedback. Automated tests catch some problems, but many judgements need a person: whether an explanation matches the code, whether every instruction was followed, or which of two working answers is more useful. Platforms recruit independent contractors with programming experience to make those judgements and explain them in writing.
The work is remote and project-based. Each platform sets its own requirements, assessments, rates and project availability — which is why acceptance and earnings are never guaranteed.
The six kinds of task
Most coding projects combine several of these.
Evaluate AI-generated code
Decide whether code actually does what was asked — including the edge cases the author forgot.
chunk([1, 2], 5) returns [] — the whole list is lost.
Compare alternative solutions
Choose between two implementations under the stated constraints, not personal taste.
O(n²) vs O(n) with O(n) memory: which fits n ≤ 100,000?
Detect incorrect reasoning
Spot explanations that sound right but contradict the code they describe.
“Preserves order” — followed by .sort().
Assess instruction following
Check every explicit requirement, including the ones no test covers.
All tests pass, but the requested Javadoc is missing.
Identify behavioural inconsistencies
Rate honesty, calibration, safety and fairness using only the evidence provided.
Confidently claims a feature the changelog says doesn't exist.
Explain and justify decisions
Write short, specific, verifiable reasoning. Often the explanation is the product.
Incorrect: [-3, -1, -2] returns 0, expected -1.
A worked example
Request: “Write chunk(items, size) that splits a list into chunks of length size. The last chunk may be shorter.”
An assistant replies with the code on the right and says the range end keeps every slice “within bounds”.
A good evaluation: “Incorrect. chunk([1, 2, 3, 4, 5], 2) returns [[1, 2], [3, 4]] — the final [5] is silently dropped, and chunk([1, 2], 5) returns []. Python slices clamp out-of-range indices, so the stated reason for the bound is wrong. Fix: range(0, len(items), size).”
It names the verdict, cites concrete inputs, checks the explanation against the code and proposes a minimal fix. That is the skill our practice labs train.
def chunk(items, size): if size < 1: raise ValueError("size must be at least 1") return [items[i:i + size] for i in range(0, len(items) - size + 1, size)]From curiosity to your first project
- 01
Discover opportunities
- 02
Check eligibility
- 03
Practise skills
- 04
Apply independently
- 05
Complete the platform's own assessment
- 06
Potentially access projects if accepted
SoftwareSideHustles.com is an independent guide. Each platform runs its own application and assessment, and decides who is accepted and which projects are offered.
Skills that matter
- Reading code you didn’t write, accurately and quickly.
- Reasoning about correctness and edge cases (empty input, duplicates, limits).
- Comparing time and memory costs against stated constraints.
- Following detailed guidelines precisely.
- Clear written English: short, specific, verifiable explanations.
What to expect honestly
- Platforms decide acceptance; many applicants are not accepted.
- Work availability varies by project, skills, location and performance.
- You are usually an independent contractor, responsible for your own taxes.
- Some listings are limited to certain countries.