Executive conclusion
GPT-3.5 wrote most of the visible code — most of it under the loosest counting rule, fewer than half of it under the strictest — and the programmer made it converge into a working game.
The programmer was substantially more than a prompt operator. They acted as product designer, architect, tester, debugger, and reviewer. Several of their contributions were essential: collision semantics, move ordering, rotation allocation, threshold handling, speed direction, sparse-row handling, and final integration fixes.
The two questions come apart sharply. GPT-3.5 supplied most of the text and was drop-in correct in 13 of 40 replies; it also truncated 11 times, described its own code incorrectly 8 times, and produced 6 replies that would have re-broken working behaviour. It never once questioned an instruction — including the one that removed wall and floor collision from the game. The reliability section puts numbers on all of that.
How the attribution was measured
All 40 exchanges in the archived transcript were compared line by line against every one of the 19 historical revisions of index.html, the commit the programmer explicitly marked “Manual fixes (not ChatGPT)”, and the final 435-line file. A 47-assertion harness was then run against the finished JavaScript to check that the behaviour claimed below is the behaviour the game actually has.
Git blame is deliberately not part of the evidence. Every commit in this repository has a single author — the programmer pasted GPT’s code in themselves — so blame attributes all 435 lines to a human and can distinguish nothing. Attribution here rests on transcript matching instead.
Upper bound: 85.4%
315 of 369 nonblank final lines have a whitespace-normalised exact counterpart in some GPT reply. Those 315 partition cleanly: 61 are nothing but a brace or bracket, 80 are comment-only, 18 are lines the programmer wrote first that GPT echoed back, and 156 are substantive statements. Anything above 85.4% is not supported by any counting rule we could find.
Lower bound: 42.3%
Take only those 156 substantive statements and you get 156 of 369 = 42.3%. Put the comments back — GPT did write them — and the middle estimate is 236 of 369 = 64.0%. Only lines the programmer used first count as echoes: in responses 37 and 38 the programmer pastes a great deal of GPT’s own earlier code back into the chat, and that is not human authorship.
Looking the other way. Counting a line as code when it ends in a brace, bracket, semicolon, comma or tag, or begins with //: GPT-3.5 emitted 974 code lines across the 40 replies with repeats, of which 367 are distinct and 208 (57%) appear in the finished game. That is 3.1 lines emitted for every line of the final file. Most of the excess is re-emission — asked to change one line, GPT-3.5 habitually returned the whole enclosing function, which is also how six replies came to silently revert working code. The survival rate is sensitive to the rule: loosen it to admit prose-like lines and the denominator inflates and the figure falls to about 44%.
The two scales used in the table
Accuracy — was this reply safe to paste?
- 100 — drop-in correct. Pasting it verbatim does what was asked and breaks nothing else.
- 70–95 — the requested change is right, but something else in the reply is not: a stray regression, a wrong literal, a missing clamp.
- 40–65 — the core idea is usable, but pasting it verbatim leaves the reported bug unfixed or introduces a new one.
- 0–35 — actively harmful. It calls functions that do not exist, truncates mid-definition, or undoes working behaviour.
Carry-over — how much of it shipped?
The share of the reply’s novel implementation still present in the final file without a later substantive correction, checked against the 19 historical revisions. Code a reply merely repeated from an earlier turn does not count as novel, which is why a reply can be perfectly accurate and still score low — response 40’s CSS is correct and was never integrated at all.
Both scales are judgements about code, made against primary sources, and are approximate to roughly ±10 points. Everything in the section above them is counted, not judged.
Across the 40 replies the two scales average 68% accuracy and 56% carry-over. Neither mean is worth much on its own — they average single-rater scores with a ±10 tolerance — which is why the distribution is given beneath the table instead.
Response-by-response analysis
Click “View exchange” to inspect the original programmer message and GPT-3.5 reply. All 40 exchanges are embedded in this page, so the feature works offline; each one reproduces the archived Word transcript to within a single separator character. Exchanges 1–36 are one continuous ChatGPT session; exchange 37 opens a second session that begins “(New Session)” and has no memory of the first, which is why the programmer has to paste their own code back in at that point.
Accuracy bands (the same thresholds shade the carry-over column):
90–100: essentially correct — 15 replies
50–89: useful but incomplete — 11 replies
0–49: materially defective — 14 replies
| # | Response topic | Accuracy | Carry-over | Assessment | Original |
|---|---|---|---|---|---|
| 01 | Full Tetris attempt | 10% | 0% | Incomplete and truncated, with invalid piece representation and rotation. | |
| 02 | Simplified falling square | 25% | 55% | Supplied the basic architecture, but collision reset the square continuously, so it never fell. | |
| 03 | First spawn-collision fix | 20% | 0% | Still returned true above the board, contradicting its own explanation. | |
| 04 | Bounds / continue rewrite | 85% | 0% | Followed the suggested algorithm, but that algorithm made walls and floor non-colliding. | |
| 05 | “Invisible walls” collision | 45% | 50% | Transcribed the programmer’s stated rule verbatim, but delivered two of the three requested walls, so it crashed at the floor. | |
| 06 | Bottom boundary guard | 100% | 60% | Correctly fixed the reported crash. | |
| 07 | Place collided piece | 40% | 55% | Checked at the wrong time and could write outside the board. | |
| 08 | First increment / undo fix | 45% | 70% | Tested position.y rather than the occupied cell’s row; the crash remained. | |
| 09 | Always undo downward move | 100% | 90% | Correct locking sequence; became the core of final move(). | |
| 10 | Draw active piece and board | 95% | 95% | Correct and almost entirely retained. | |
| 11 | Seven shapes attempt | 85% | 85% | Shapes were valid, but the reply was cut off inside the L, having promised all seven. | |
| 12 | Add L shape | 100% | 95% | Correct full shape collection. | |
| 13 | Random initial piece | 85% | 45% | Correct selection logic, but the proposed start() wrapper was not retained. | |
| 14 | newTetromino() refactor | 100% | 90% | Sound structure and heavily retained. | |
| 15 | Ignore empty cells | 30% | 20% | The requested skip was correct, but the full replacement lost the above-board guard and was unsafe. | |
| 16 | Game-over return value | 45% | 55% | Top test was before undoing the move; it also omitted the ordinary true return. | |
| 17 | Top check after decrement | 55% | 65% | Improved ordering but still checked only after writing to the board. | |
| 18 | Top check before board write | 100% | 90% | Correct final ordering. | |
| 19 | update() checks move() | 60% | 45% | Correct return check, but its full version removed timing and would move every animation frame. | |
| 20 | Left / right handler | 100% | 90% | Correct and mostly retained. | |
| 21 | Register keyboard handler | 40% | 10% | The registration line was correct, but it invented nonexistent initialization functions. | |
| 22 | Down key sets dropping | 100% | 90% | Correct incremental change. | |
| 23 | Fast-drop timing | 100% | 70% | Correct timeout / animation-frame structure; literal timing was later changed. | |
| 24 | Reset dropping on new piece | 40% | 5% | Contained the requested line, but used the wrong tetrominos name and incompatible state initialization. | |
| 25 | Post-drop adjustment period | 20% | 15% | Reversed the requested state transition and called nonexistent saveTetromino(). | |
| 26 | Rotation functions | 45% | 65% | Correct transformation formula, but it crashed because output rows were allocated too late. | |
| 27 | Allocate rotation rows | 100% | 95% | Exact and effective correction. | |
| 28 | Bind Z to rotate | 60% | 45% | Correct action, but added another listener rather than modifying the existing handler. | |
| 29 | Clear filled rows | 70% | 85% | Mostly good, but new sparse rows caused phantom line clears until manually fixed. | |
| 30 | Score calculation | 100% | 95% | Correct implementation of the requested score table. | |
| 31 | Display score | 100% | 95% | Correct and retained. | |
| 32 | Level and speed system | 75% | 45% | Speed loop worked; level-up failed when multiple rows crossed a threshold. | |
| 33 | Multi-row threshold | 100% | 95% | Correct fix, even without the suggested flag. | |
| 34 | Display level and rows | 65% | 65% | Correct DOM lines, kept verbatim but relocated; it also injected an unrequested level-up call ahead of scoring, and cut off mid-sentence. | |
| 35 | Exponential speed | 55% | 20% | Right shape, but it silently changed the base to 1.1, dropped the 10 ms clamp, and then claimed the clamp was there. | |
| 36 | Use 0.9 and clamp | 40% | 35% | Used division instead of multiplication, making higher levels slower. The programmer corrected it. | |
| 37 | Initial mobile buttons | 70% | 60% | Useful HTML, but it initially called undefined helper functions. | |
| 38 | Handler / helper refactor | 30% | 0% | Incomplete, redundant, and abandoned by the programmer. | |
| 39 | Buttons call existing handler | 100% | 95% | Simple, correct workaround; it survives in the final game. | |
| 40 | Flexbox button CSS | 95% | 0% | Correct CSS, but it was never integrated. |
How reliable was GPT-3.5?
Attribution asks who wrote the surviving text. This section asks a different question: across the 40 replies, how often was GPT-3.5 right, how did it fail when it was wrong, and how much of what it produced was wasted. Everything here except the first tile is counted from the transcript and the 19 historical revisions rather than estimated; the first tile derives from the judged accuracy column of the table above.
How it failed, and how often
| Failure mode | Count | Responses | What it looked like here |
|---|---|---|---|
| Ignored a stated constraint | 8 | 1, 2, 5, 8, 16, 25, 35, 36 | The programmer asked for a 10 ms floor on the drop interval and #35 dropped it entirely; #36 restored the clamp but substituted division for multiplication, so higher levels ran slower. |
| Described its own code wrongly | 8 | 1, 2, 3, 5, 28, 32, 35, 36 | #3 explains that the function “returns false because the tetromino is not colliding” directly beneath code that returns true. #35 claims a 10 ms cap it never implements; #36 does clamp correctly but describes its own division as making the interval “increase faster for higher levels”, when it makes higher levels slower. |
| Truncated mid-output | 11 | 1, 2, 8, 11, 12, 15, 16, 17, 26, 34, 38 | In #1, #11 and #38 the cut lands inside the code; the other eight lose only the trailing explanation. #11 promises seven shapes and stops at // L-shaped tetromino [ [0, costing a whole turn to finish. |
| Rewrote more than it was asked to | 6 | 15, 19, 21, 24, 25, 28 | Asked for a one-line skip, #15 returned a whole new checkCollision() that silently dropped the above-board guard established ten turns earlier in #5. |
| Put correct logic in the wrong order | 5 | 7, 16, 17, 26, 34 | The game-over test took three turns to land, moving from after the board write, to after the decrement, to finally before both. #26 allocated rotation rows inside the loop that writes them. |
| Called something that did not exist | 5 | 10, 21, 24, 25, 37 | createBoard() and main() in #21, saveTetromino() in #25, tetrominos for tetrominoes in #24, three undefined button handlers in #37 — and BLOCK_SIZE in #10, which the programmer liked enough to define rather than reject. |
| Re-introduced a fixed bug | 3 | 15, 24, 25 | #24 reverted the random spawn column back to a hard-coded {x: 3, y: -2} — twenty-two turns after GPT itself introduced that randomness in #2. |
| Redundant or self-cancelling code | 3 | 28, 34, 38 | #38’s handler calls moveLeft(), which already reverts its own collision, and then reverts it again. |
| Correct, but never used | 3 | 13, 38, 40 | #40’s flexbox CSS is correct and was never added to the page — the buttons are still unstyled in the shipped game. |
Fourteen replies carry no failure tag at all: 4, 6, 9, 14, 18, 20, 22, 23, 27, 29, 30, 31, 33, 39. The pattern is consistent — GPT-3.5 was reliable when the request was a self-contained, well-specified piece of code, and unreliable when the request required holding the rest of the file in mind.
Where the 40 turns went
Eight turns on one function
Collision semantics — responses 3, 4, 5, 6, 7, 8, 9 and 15 — took a fifth of the entire session, and checkCollision() was still one of only two functions the programmer had to repair by hand afterwards. Game-over ordering took three turns; the level and speed system took five; the mobile buttons took four, of which one was abandoned.
Against that, six subsystems landed in one or two turns: drawing, the piece table, rotation, line clearing, scoring, and the left/right handler.
Who supplied the fix
15 of the 40 programmer messages — 4, 5, 7, 8, 9, 14, 15, 17, 18, 25, 27, 33, 35, 36 and 39 — named the change at the level of a line, a condition, or an expression — “The if (position.y < 0) check should probably go before the for-loops, but after position.y--”, or “Could you use Math.pow(0.9, level -1) instead? Also I think you should use Math.max instead of Math.floor here, to clamp it to a minimum of 10 ms.” The remaining 25 stated a goal or reported a symptom.
Every hard bug that was closed inside the conversation was closed by a message of that kind. The last two were closed outside it, by the programmer editing the file directly. That is the clearest signal in the transcript about what this collaboration actually required.
What survived, function by function
| Function in the final game | Lines | Traceable to GPT | Turns spent on it |
|---|---|---|---|
keyboardHandler() | 47 | 100% | 5 |
updateHighscore() | 24 | 100% | 2 |
draw() | 22 | 100% | 1 |
rotateClockwise() | 20 | 100% | 2 |
incrementClearedRows() | 17 | 100% | 3 |
newTetromino() | 8 | 100% | 3 |
rotateCounterClockwise() | 7 | 100% | 2 |
move() | 32 | 94% | 6 |
clearFilledRows() | 30 | 93% | 1 + manual fix |
update() | 15 | 87% | 3 |
getLevelSpeed() | 7 | 86% | 3 |
checkCollision() | 25 | 84% | 8 + manual fix |
handleClearedRows() | 5 | 60% | — |
| HTML shell, piece table, game state | 110 | 63% | — |
One caveat on the 100% rows. The percentages count a final line as GPT’s if it appears in any GPT reply, and in responses 37 and 38 the programmer pastes their own code into the chat, which GPT then echoes back. keyboardHandler()’s fast-drop plumbing — gameTimeout, clearTimeout(gameTimeout); update(); — reaches GPT only that way, so its true figure is nearer 86% than 100%. Two other rows are GPT’s text carrying the programmer’s design: the 40/80/120/400 score table in updateHighscore() and the rotate-three-times trick in rotateCounterClockwise() were both specified in the prompt.
The two functions the programmer had to edit by hand — checkCollision() and clearFilledRows() — are also the two whose bugs depended on something outside the function: a sparse array created elsewhere, and a guard ordering that only matters for a piece straddling the top edge. Those are precisely the bugs a single-function prompt cannot surface.
What the programmer contributed
In 15 of the 40 messages the programmer supplied the invariant or the algorithm that made the next response useful — not merely a report that something had gone wrong. Those 15 messages close every hard bug GPT closed. The last two — the sparse-row phantom clears and the collision-guard ordering — the programmer closed alone, in commit 2d6eb2e.
Prompting and diagnosis
- Reduced the task after the first answer failed.
- Prescribed the bounds-and-continue rewrite that broke wall and floor collision — the one intervention that cost turns rather than saving them.
- Defined infinite space above the board and solid boundaries elsewhere.
- Supplied the move sequence: increment, test, undo, then save.
- Diagnosed exceptions down to the exact array access.
- Specified that empty tetromino cells must be ignored.
- Corrected game-over ordering across several turns.
- Identified rotation allocation and multi-line threshold errors.
- Abandoned an unsuccessful refactor and chose a simpler design.
Direct and integration work
- Created the surrounding HTML shell, and defined
BLOCK_SIZEafter GPT began using it undeclared. - Fixed sparse rows causing phantom line clears.
- Hardened collision-boundary ordering.
- Changed the level formula from division to multiplication after GPT inverted the direction in response 36.
- Caused, then repaired, the counter-clockwise regression — using, in effect, the restoration GPT had already supplied in a reply they had otherwise rejected. See below.
- Added layout, debug controls, and the testing/integration workflow.
One correction to the record. The counter-clockwise regression is usually told as GPT breaking something. It is the opposite. Commit 3d39e24, which integrated response 39’s button HTML, silently emptied the keyCode == 90 branch of the keyboard handler; the emptied branch is visible in the code the programmer pastes into response 38. GPT’s reply to 38 restored that branch correctly, and the seven lines it produced are, apart from indentation, exactly what commit 9c97474 put back the next day — but the programmer discarded response 38 for its other faults and put the fix back by hand. The regression was a human integration slip, and the repair had already been offered.
Is the final game working?
Yes — as a minimal Tetris-like game. A harness of 47 assertions was run against the finished JavaScript with a stubbed DOM, a controllable clock and a seeded random source. All 47 pass — including the ones that pin down the limitations on the right. It ships with this report — node docs/verify/tests.js — so every claim below can be re-checked against the game as it stands.
Confirmed by the harness
- A 10 × 20 board and seven square piece matrices, each with exactly four filled cells.
- Collision against the floor, both walls, settled blocks, and empty space above the board — including empty piece cells sitting beside a wall.
- Falling, locking, respawn, and a game-over alert when a piece locks above the board — though see the limitation about what happens next.
- Row clearing for one, two stacked, and four rows at once, with no phantom clear from the freshly created top row.
- Levels rising every five rows, including when a single clear crosses the threshold, and
getLevelSpeed()returning exactly1000 × 0.9level−1ms clamped at a 10 ms minimum. (The realised drop period adds one animation frame per row on top of that.) - Rotation that is a genuine clockwise / counter-clockwise transform, is identity after four turns, reverts when blocked, and never mutates the shared piece table.
- Keyboard input, with the handler registered exactly once, and all five mobile buttons routing real key codes into that same handler.
Remaining limitations
- Space does not drop a piece; only Down is handled — the one control from the opening prompt that never got implemented.
- The post-fast-drop adjustment window is 20 ms, which is not enough time to use. The 20 is the programmer’s literal — GPT supplied the 10 ms that was asked for in response 23.
- Response 40’s mobile-button CSS was never applied, so the buttons are unstyled.
- Game over is not final: it alerts and stops the loop, but a single press of Down restarts it with the piece still hovering.
- Holding Down on a piece that has already landed keeps re-entering the grace branch, so it never locks.
- There is no restart — nothing resets the board, score or level, so a second game means reloading the page.
- The handler never calls
preventDefault(), so the arrow keys scroll the page while you play. - A debug button remains visible; each press awards score, increments the cleared-row counter, and levels the game up — and so speeds it up — every fifth press.
- Pieces spawn at
x = 0…7, so nothing ever spawns against the right wall. - Scoring is 40 / 80 / 120 / 400 per level, not the classic 40 / 100 / 300 / 1200 — this is what the programmer asked for in #30, not a GPT error.
- There are no wall kicks (a blocked rotation is simply refused), no rotation centring (a 180° turn shifts the I-piece one column inside its box), no next-piece preview, no hold or pause, and no seven-bag randomiser.
Could GPT-3.5 have done it alone?
Probably not without a knowledgeable collaborator. Note what “alone” means here: this model could not run the game, see the canvas, read a stack trace, or edit a file. Every symptom it worked from had to be observed and typed by the programmer, so the bands below describe a model plus a no-feedback-loop harness, not capability in isolation. Both figures are judged — our own credences, from a sample of one, about re-runs that never happened. They are not measurements and should not be quoted as any. “Success” would mean passing the same runtime assertions used above.
One prompt, paste the answer, obtain a working game meeting the requested controls. Response 1 was truncated mid-function and used a piece representation its own draw() could not read, so this run scored zero on that path.
Several generic “it’s broken, fix it” prompts, with no knowledgeable diagnosis and no algorithmic hints. The transcript gives almost no direct evidence here, because the programmer almost never sent a hint-free prompt. What it does show is that two of the three replies that followed the barest reports — #6’s stack trace and #8’s exception — still needed another turn.
Observed, not estimated. With this programmer’s targeted testing, diagnoses, design decisions and integration it worked — in 40 turns, three hand-written repairs, and about four and a half hours. The two cells to the left are guesses; this one is the run we actually have.
The direct evidence is strong: the first response was cut off mid-function by the output-length limit and the part that did arrive used a piece representation its own draw() could not read; the simplified second response did not animate correctly; and several core functions failed repeatedly until the programmer supplied the relevant invariant or repair strategy.
It does not all run one way. Response 4 broke wall and floor collision precisely because it faithfully implemented the bounds-and-continue algorithm the programmer had just prescribed, and responses 5 and 6 were spent recovering from that. The programmer’s guidance was decisive, but it was not uniformly corrective — and GPT-3.5 never once said so.
What this is and is not evidence about. This is one session with the original ChatGPT research preview — GPT-3.5, three days after its public launch on 30 November 2022, with no tools, no file access, no ability to run what it wrote, and a hard output-length limit it hit eleven times. Everything here is a claim about that model on this task on those three days in December 2022. It says nothing about how a current model would do, and a single transcript cannot establish a rate for GPT-3.5 either.