Parrot Cart developersGet a Parrot Cart

Testing

On your computer

test/run.sh                    # the template's plugin and every example
test/run.sh hello sound        # some of them
NO_MAME=1 test/run.sh          # without the MAME step

test/run.sh compiles each plugin’s main.c for your computer and runs it against models of everything on the cart (test/host/host.c):

The modelWhat it doesWhat it catches
the screen kernelruns each list a frame after it’s sent, sends HELLO and ACK reports, and reports the controller from a scripta malformed list, a list sent too early, three colors in an 8-pixel run, and a screen where the SDK and the console don’t match
the frame kernelruns one list each TV frame and adds up its time from the kernel’s costsa list that runs past its frame or its vertical blank, and bad commands
the carda disk image, read through the SDK’s file system (FAT16, FAT32, exFAT)wrong reads and saves that get lost. Afterward, macOS’s fsck checks the image
the menua launch block for a file on the imagea plugin that can’t open its file
USBa pad, keyboard, mouse or listening computer, from the scripthow the plugin handles them (USB itself isn’t tested)
the PSRAM8 MB of ordinary memorythe plugin’s arithmetic

Then the test plays the plugin’s lists to the SDK’s real Z80 kernel on MAME’s ColecoVision (tools/mame-replay.sh, headless). MAME’s picture has to match the model’s pixel for pixel, and for the frame kernel, no list can make a frame start late.

That step needs mame, sjasmplus and the ColecoVision BIOS. MAME calls the BIOS coleco.zip, and the kit doesn’t include it, so bring your own. Put it in a folder outside the plugin’s repository and name that folder in MAME_ROMPATH:

MAME_ROMPATH=$HOME/mame/roms test/run.sh

You can also put it where MAME already looks: the rompath in its mame.ini, or, with no mame.ini, a roms folder in the folder you run the test from. If MAME can’t find it, test/run.sh says so and skips the MAME step, as NO_MAME=1 does.

To test your plugin this way, add a block for it to test/run.sh by copying the hello one. A plugin made by tools/new-plugin.py already has a block. You can also run it by hand. The script’s events are listed at the top of test/host/host.c, and the USB ones are in docs/features.md:

test/host/build.sh src /tmp/myplugin
PC_OUT=/tmp PC_END=3000 PC_SCRIPT="500:joy=up 900:key=5" /tmp/myplugin
open /tmp/screen.png; cat /tmp/log.txt

The card tests need macOS (hdiutil), and the rest also run on Linux.

To play your plugin instead of scripting it, use the simulator. It runs the same models in a window, with sound, the keyboard, mouse and game pads, your computer’s MIDI ports, and a folder as the SD card (docs/simulator.md). Its headless mode takes the same scripts, plus a few extra words for the keyboard, mouse and pads, and sim/test.sh runs every example that way.

How loud

A plugin should sound about as loud as an ordinary ColecoVision cartridge with the TV at the same volume. The cart’s first arcade plugins were far too quiet next to one. Compare the two in MAME, recorded the same way:

# a cartridge you own, 60 seconds of its title and attract mode
SDL_VIDEODRIVER=dummy mame coleco -rompath "$MAME_ROMPATH" -cart cart.col \
    -video none -videodriver dummy -window -nomaximize -sound none \
    -nothrottle -skip_gameinfo -seconds_to_run 60 -wavwrite cart.wav
# your plugin's lists from its test/run.sh run, on the same machine
MAME_WAV=plugin.wav tools/mame-replay.sh "${TMPDIR:-/tmp}/parrotcart-test/myplugin/lists.bin" /tmp/replay 60
python3 tools/wavlevel.py cart.wav plugin.wav

-sound none keeps MAME quiet on the computer, and -wavwrite still records. wavlevel.py gives each file’s level in dBFS, leaving out its silent seconds. Get the plugin’s rms within a few dB of the cartridge’s. A peak at 0 means the sound clips. The lists are only as long as the test’s script, so give it a script that plays the game’s sound for a while. Then listen on a TV, which MAME can’t stand in for.

Ordinary cartridges measured this way give an rms of -20.5 to -23.9 dBFS (Donkey Kong, Smurf, Lady Bug and Mr. Do!), about -22 on average. Aim within 3 or 4 dB of that and not above it.

What you can only test on a console

The models assume a console and devices that behave perfectly. These things only get tested when your plugin runs on a ColecoVision with the cart:

  • Starting the plugin and going back to the menu, including the menu copying the plugin into flash, RESET, and switching the console off and on while the plugin runs.
  • USB devices, since every one is different. Try hubs, how pads start up, and a computer’s first “Allow this accessory?” prompt.
  • A real SD card, with its start-up time and slow writes, and what happens when someone pulls it out.
  • The PSRAM chip itself.
  • How the sound really sounds, how loud it is next to a cartridge, and how the picture looks on a TV, where colors bleed and one-pixel lines shimmer.
  • The bus. A frame plugin’s note can say how many addresses the bus loop refused (cvf_bus_counts()). A few are normal, and many point at the cart’s seating or the console.
  • Memory, meaning the plugin’s real heap and stack. The Mac’s are bigger.
  • Speed. The Mac runs your loop a thousand times faster, so a loop pass that takes 30 ms on the cart only shows up as a sluggish screen on the console.

A test plan for the examples on a console

Copy the ten .CVP files to the card’s /PLUGINS folder, and put a text file or two anywhere on the card. Check these steps for every plugin first:

#DoExpect
AGo to Home, then the Plugin ManagerEach plugin is listed with its icon in magenta, its name and its version. Text Viewer, which needs a file, has the plug icon
BPress A on it, then *Its details (version, developer Parrot Cart or yours, ID, what it uses), then its README, 30 columns wide, with pages turning
CPress B, then AThe copy into flash counting up the first time, then the plugin’s screen within a second or two
DPress RESETThe menu, back in the Plugin Manager, with the plugin’s note on the bottom line
EStart it again. Switch the console off, wait 5 seconds, and switch it onThe menu, as after a cold start

Then check each plugin:

PluginDoExpect
Hello (HELLO.CVP)Use the stick, both fire buttons and each keypad keyEach one is named on the TV while you hold it, and PRESSES counts them. After RESET, the menu says Hello: N presses
Screen ExampleWatch it, then press fireThe ball moves smoothly inside its black box, with no trail and no flicker in the parts that don’t move. Fire changes its color. The 15 swatches are 15 different colors. The bricks are red on gray, and the coins are yellow
Sound ExamplePress keypad 1 to 8, move the stick up and down, and press left fireA C major scale, in tune, with each note stopping when you let go of the key. The stick moves the octave up and down. Left fire plays a drum hit that fades in a third of a second. There are no clicks or stuck notes, and it’s silent after RESET
Game Pad ExampleStart with nothing plugged in. Then plug in a powered hub with a pad, press everything, move both sticks to their corners, and pull the pad out and put it backNO PAD YET at first, then SWITCH PRO or HID PAD within a few seconds. Each button lights its name, under the right name for where it is (the bottom face button is SOUTH). Each dot follows its stick, rests near the center (numbers near 0) and reaches about +-2047 at the edges, with up being positive. DIRECTIONS follows the d-pad and the left stick. Unplugging goes back to NO PAD YET without freezing, and plugging in works again
Keyboard ExamplePlug a keyboard and a mouse into the powered hub. Type, press Enter and Backspace, move the mouse and clickKEYBOARDS 1 MICE 1. Letters appear as you type, with none lost or doubled. Enter moves the lines up. The pointer follows the mouse to all four edges of the blue field, and turns red (left button) or green (right button)
MIDI Out ExamplePlug the cart’s USB-C into a Mac and allow the accessory if asked. Open Audio MIDI Setup or GarageBand, and press keypad 1 to 8The TV says A COMPUTER IS LISTENING, and the Mac lists Parrot Cart MIDI Example. Each key plays a note that stops when you let go, and SENT counts two for each key. With nothing plugged in, the TV says PLUG THE CART INTO A COMPUTER, and the keypad still counts without hanging
SD Card ExampleRun it, press RESET, run it again, then look at the card on a computerThe top folder’s first 8 entries with their sizes, the first file’s first characters, and RUN 1, SAVED AND READ BACK, then RUN 2. The menu’s note gives the run. On the computer, /SDCARD.SAV is 512 bytes and the card checks clean. Try a FAT32 card and an exFAT card. If it can’t read the card, it says so in red
Text ViewerIn the card’s folders, press A on a .txt file (it has the plugin’s green icon). Move the stick down and up, hold left fire and move the stick, then press RESETThe file’s name and text, wrapped at 30 columns. The stick scrolls by a line, and by a page with left fire held. Back in the menu, you see the same folder with the same file selected, and Read NAME on the bottom line. In the Plugin Manager, its details say it starts with a file, and A there doesn’t start it. A name with an accented letter still opens
Frames ExampleWatch for a minute, and steer with the stickEight color bars appear within a quarter of a second. A round white ball moves smoothly, with no tearing, no stray blocks left behind and no stutter. There’s a short beep at each bounce and silence between, with no drone, and the stick turns the ball
PSRAM ExampleRun itID 0D 5D ..., the bar filling twice, then ALL 8 MB READ BACK RIGHT and two times in milliseconds. Write the times down, because they’re the PSRAM’s real speed

If one fails, look first at the menu’s note after RESET, and at the handoff trace on the menu’s USB serial (t).

This page is docs/testing.md in the starter kit.