Before I Trained Anything, Ace Made Me Build a Filing Cabinet

Ace handing me the documentation plan

Once I decided to try building my own rights-cleared pen-and-ink LoRA, I assumed the first step would be finding pictures. Ace had other plans. He made me start with a folder called AI_LoRA_Training. I thought I needed a few simple subfolders for images, descriptions, article notes, and sources. Again, I was wrong. Four folders became eleven.

We built the file structure to stay organized. The original images needed one place. The copies prepared for training needed another. Rights information, captions, training settings, test results, generated images, and blog notes all needed their own folders. Apparently, before I could teach an AI to draw, I needed somewhere to file the paperwork.

The insane folder structure

Why the File Structure Mattered

Original vs. trainer ready image of Hond snuffelt aan een spiegel

The goal was not to make the project look impressively complicated. The goal was to keep the different parts of the experiment from getting mixed together.

Otherwise, I could easily end up with a folder full of mysterious downloads named things like:

final_final_2_reallyfinal.jpg

The folder structure makes it look very official, and it helps me remember where everything belongs.

One of the first rules was that the original downloaded image should never be edited.

If I cropped it, cleaned it, resized it, removed a border, converted it to black and white, or changed the file format, that new version would go into the processed-images folder. The original would stay exactly as it was when I downloaded it.

If something went wrong later, I could return to the original instead of trying to remember what I had changed. It also meant I could compare the training copy with the file provided by the museum, archive, or other source.

The same idea applied to rights information. A link alone was not enough. Webpages change. Files move. Language gets updated. For every image I accepted, I wanted a record of where it came from, who created it, what the source said about its rights status, and when I accessed that information. Copies or screenshots of the relevant rights statements would be stored with the source records in a handy dandy spreadsheet.

The devil is in the details

Planning Before Collecting

We still had some basic questions to answer: which base image model to use, how many images we needed for a useful first test, how the images should be captioned, and what evidence was enough to accept an image into the dataset.

The next step was to establish the dataset rules before collecting the dataset itself, so I wouldn’t make up the rules just because I found an image I really wanted to use.

Sources consulted

Pew Research Center, When Online Content Disappears Longpre et al., The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AIGebru et al., Datasheets for Datasets Hugging Face, Create a Dataset for Training