Coding Test Maker (Programming Tests With Auto-Graded Code)

About the Coding Test Maker (Programming Tests With Auto-Graded Code)

Write coding problems, send candidates one link, and mark every answer in your own browser against tests they never see. Each problem asks for a program that reads its input and prints its output, a function that returns the answer, or an SQL query; candidates answer in Python 3, JavaScript or TypeScript (or SQL), in a real code editor, and run your sample tests as often as they like before they send their answers. Add multiple-choice questions for theory, a time limit for the whole test, and points for each problem.

There is no server. The link carries the problems and the sample tests only — never the hidden tests, your solution or the right answers, because anything inside a link can be read by whoever holds it. Each candidate finishes with a response code encrypted for your browser alone; you paste the codes in, and every answer runs against every test in an isolated sandbox: the points of a problem are shared out by the tests it passes, with a rank list, each candidate’s test-by-test results, their code, and flags for pasted code, time away from the page and code that is much the same as another candidate’s.

Write your own reference solution and the page fills in every expected answer from it. Outputs can be compared exactly, line by line in any order, as numbers within a tolerance, or by a checker you write yourself for problems with more than one right answer. Without a pass the free preview builds and tries the test in full and marks the first responses of a batch with their totals; the candidate link, the whole batch with every test’s result and the similarity flags, and the Excel, CSV and PDF reports unlock with a Premium pass.

Common uses

• A programming class’s weekly lab test, taken at home and marked against hidden tests in minutes.
• A first technical screen for a hiring round: three problems and a few questions, one link for every applicant.
• A practice challenge for a coding club: candidates run the sample tests and nothing comes back.
• An SQL test for a databases course, with each test changing the data under the query.

Why the hidden tests are not in the link

A link can be read by whoever holds it, however it is packed. So the candidate link carries what a candidate should see — the statements, the starting code and the sample tests — and nothing they should not: the hidden tests, your reference solution, your checker and the right answers of the questions stay in this browser and in your author file.

Answers come back the other way: when a candidate finishes, their browser encrypts their code and choices with a key in the link that only your browser can open (ECDH P-256 and AES-256-GCM, from the browser’s own Web Crypto). Nobody else who sees a response code can read it, and a changed code does not open. It does not prove who sent it — anyone with the link can make a response — so check the names, and supervise a test whose marks matter.

How answers are run and marked

Python, JavaScript and TypeScript run in MySmartCoPilot’s code sandbox: a separate, isolated address that cannot reach this page, your files or your cookies. Python is CPython compiled to WebAssembly (Pyodide), JavaScript runs in the browser’s own engine, TypeScript is turned into JavaScript first. SQL runs in SQLite (sql.js) in a worker of this page, with a fresh database for every test.

• Read the input, print the output: the test’s input is the program’s standard input (Python’s input(); readline() and INPUT in JavaScript), and what it prints is compared with the expected output, spaces at the ends of lines and blank lines at the end ignored.
• A function: each test calls the function with its JSON list of arguments and compares the value it returns; numbers can be compared within a tolerance and lists in any order.
• An SQL query: each test makes your tables, runs its own SQL, then the query; its last result set is compared with the expected rows, in order or not.

Every test has its own time limit; a program that runs longer, stops with an error or gives another answer fails that test, with the reason shown. Each candidate’s first Python run gets an interpreter nothing has run in, so one candidate’s code cannot change another’s results. A problem’s points are its points × tests passed ÷ tests, rounded to a tenth.

What the results show, and what they cannot

The rank list orders candidates by score (the faster finish first on a tie) with each problem’s points and tests passed, the questions’ points and the time used. Each candidate’s results list every test with its result, its time and why it failed, and their code.

The notes are things to look at, not verdicts: how many times the candidate pasted into the editors and how much, how often they left the page, and pairs of candidates whose code for a problem is much the same (compared as the Code Plagiarism Checker does, with renamed variables and moved code still matching). Shared code and pasting are reasons to look closer, not proof of copying, and a browser cannot see a second screen, a phone or another person: a test whose marks matter should be supervised.

Limitations

• Python 3, JavaScript, TypeScript and SQL (SQLite) only; Java, C and C++ cannot run in the sandbox yet. Up to 20 problems, 60 questions and 50 tests a problem.
• Python programs can use its standard library only (no pip packages), and JavaScript has no Node.js modules (no require or fs).
• Up to 500 responses at a time. Marking runs every test of every answer on your device, so a large class takes a while: about a second a test for Python on an ordinary laptop, much less for JavaScript and SQL.
• Results are self-reported: candidates send their own response codes. The page records time, pastes and time away from it, and nothing else; it is not invigilation.
• A long test makes a long link: above about 8,000 characters some chat and e-mail apps cut it, so send the link as a file.
• Tests and their keys are kept in this browser. Clearing the site’s data without the author file loses the key, and responses to that test can no longer be opened.

Privacy

On this device; its files download once

There is no server in this tool. The test lives in the part of the link after the “#”, which browsers never send to a server; your tests, their hidden tests, solutions and keys stay in this browser (and in the author files you save). Candidates’ code runs in an isolated sandbox inside their own browser and is kept there until they finish; their answers reach you only as the encrypted response code they send you themselves, and it records their name, their code and choices, the time they took, how often they left the page and how much they pasted, which they are told before they start.