Development

API

DOMjudge comes with a fully featured REST API. It is based on the CCS Contest API specification to which some DOMjudge-specific API endpoints have been added. Full documentation on the available API endpoints can be found at http(s)://yourhost.example.edu/domjudge/api/doc.

DOMjudge also offers an OpenAPI Specification ver. 3 compatible JSON file, which can be found at http(s)://yourhost.example.edu/domjudge/api/doc.json.

Bootstrapping from Git repository sources

The installation steps in this document assume that you are using a downloaded tarball from the DOMjudge website. If you want to install from Git repository sources, because you want to use the bleeding edge code or consider to send a patch to the developers, the configure/build system first has to be bootstrapped.

You can either spin up a development Docker container or install locally.

The local install requires the GNU autoconf/automake toolset to be installed, and various tools to build the documentation.

On Debian(-based) systems, the following apt command should install the packages that are required (additionally to the ones already listed under domserver, judgehost and submit client requirements):

sudo apt install autoconf automake bats \
  python3-sphinx python3-sphinx-rtd-theme fontconfig python3-yaml \
  latexmk yarnpkg texlive-latex-recommended texlive-latex-extra tex-gyre

For Fedora use:

sudo dnf install git autoconf automake bats \
  sphinx-build python3-sphinx_rtd_theme latexmk yarnpkg texlive-cmap \
  texlive-metafont texlive-tex-gyre texlive-fncychap texlive-wrapfig \
  texlive-capt-of texlive-framed texlive-upquote texlive-needspace \
  texlive-tabulary texlive-parskip texlive-oberdiek texlive-makeindex \
  texlive-ellipse texlive-pict2e texlive-collection-fontsextra

When this software is present, bootstrapping can be done by running make dist, which creates the configure script, downloads and installs the PHP dependencies via composer and generates documentation from RST/LaTeX sources.

Maintainer mode installation

DOMjudge provides a special maintainer mode installation. This method does an in-place installation within the source tree. This allows one to immediately see effects when modifying code.

This method requires some special steps which can most easily be run via makefile rules as follows:

make maintainer-conf [CONFIGURE_FLAGS=<extra options for ./configure>]
make maintainer-install

Note that these targets have to be executed separately and they replace the steps described in the chapters on installing the DOMserver or Judgehost.

Running several checkouts side by side

When working on multiple branches at the same time, for example using git worktree or jj workspace, each checkout can have its own in-place installation. No extra flags are needed: make maintainer-conf derives an instance name from the name of the directory the source tree lives in, and uses it for

  • the webserver and PHP-FPM configuration file names installed into /etc, and the name passed to a2enconf, or to a2ensite for a named instance, whose Apache configuration is a VirtualHost of its own;

  • the PHP-FPM pool name and its listening socket;

  • the nginx upstream name and the nginx variable holding the HTTPS flag, both of which are global to nginx: a duplicate makes nginx refuse to start;

  • the /etc/sudoers.d file name;

  • the default database name and database user.

In a linked git worktree, credentials are copied from the main checkout’s etc/*.secret files where those exist, so that one set of credentials works for all worktrees:

  • the database user and password, e.g. for a database GUI; only the database name differs. Removing an instance with dj_setup_database uninstall keeps the user as long as other databases still use it;

  • the initial admin password of the web interface;

  • the judgehost password, so one judgedaemon etc/restapi.secret can list the API URLs of several worktrees.

The directory name is lowercased, characters outside [a-z0-9-] are replaced by dashes, and the result is truncated to 24 characters. A checkout directory named domjudge keeps all the historic defaults, so existing installations are unaffected.

Each instance also gets its own base URL, http://<instance>.localhost/, serving DOMjudge from the root rather than from a /domjudge subpath. Distinct hostnames are used rather than distinct ports or paths on purpose: browsers do not scope cookies by port, and DOMjudge does not scope its session cookie to the base path, so two instances reachable under the same hostname would keep invalidating each other’s sessions. On systems using systemd, nss-myhostname resolves any *.localhost name to 127.0.0.1, so no /etc/hosts entries are needed.

A named instance takes its server_name and its listening port from the base URL, but only when the URL names a port explicitly: an https base URL still yields listen 80, since the generated server block speaks plain HTTP and terminating TLS is left to you.

To override either value, pass it explicitly; CONFIGURE_FLAGS is appended last, so it takes precedence:

make maintainer-conf CONFIGURE_FLAGS="--with-instance-name=myname \
    --with-baseurl=http://myname.localhost/"

Then continue as usual, and set up a database for this instance:

make maintainer-install
sudo make inplace-postinstall-nginx     # or -apache
./sql/dj_setup_database -u root [-r|-p ROOT_PASS] install

A few things to be aware of:

  • The inplace-postinstall-* targets refuse to run when a file they would install already exists with different contents, so it belongs to a different source tree or was not put there by an in-place install at all. This catches two checkouts whose directory names reduce to the same instance name; give one of them an explicit --with-instance-name.

  • If webapp/.env.local contains a DATABASE_URL line, it takes precedence over etc/dbpasswords.secret. Changing only the secrets file then moves dj_setup_database and the tooling to the new database while leaving the webapp on the old one. Keep the two in sync, or remove the DATABASE_URL line.

  • misc-tools/check-systemd-sandbox may suggest a ReadWritePaths= override for your webserver or PHP-FPM service. It goes into a drop-in file named after the instance, domjudge-<instance>.conf, so the overrides of several checkouts do not overwrite each other.

  • The judgehost side is deliberately not parameterised: the domjudge-run-* users, the cgroups and the chroot directory are shared by the whole machine. To judge against a particular instance, point that judgedaemon’s etc/restapi.secret at it.

Makefile structure

The Makefiles in the source tree use a recursion mechanism to run make targets within the relevant subdirectories. The recursion is handled by the REC_TARGETS and SUBDIRS variables and the recursion step is executed in Makefile.global. Any target added to the REC_TARGETS list will be recursively called in all directories in SUBDIRS. Moreover, a local variant of the target with -l appended is called after recursing into the subdirectories, so recursion is depth-first.

The targets dist, clean, distclean, maintainer-clean are recursive by default, which means that these call their local -l variants in all directories containing a Makefile. This allows for true depth-first traversal, which is necessary to correctly run the *clean targets: otherwise e.g. paths.mk will be deleted before subdirectory *clean targets are called that depend on information in it.

Debugging and developing

While working on DOMjudge, it is useful to run the Symfony webapp in development mode to have access to the profiling and debugging interfaces and extended logging. To run in development mode, create the file webapp/.env.local and add to it the setting APP_ENV=dev. This is automatically done when running make maintainer-conf when the file did not exist before. For more details see the Symfony documentation.

The webapp/.env.local file can also be used to overwrite the database version. This is needed to automatically generate migrations based on the current database compared to the models. To set the correct version, add a line to webapp/.env.local with the following contents:

DATABASE_URL=mysql://<user>:<password>@<host>:<port>/<database>?serverVersion=<version>

Replace the following:

  • <user> with the database user.

  • <password> with the database password.

  • <host> with the database host.

  • <port> with the database port, probably 3306.

  • <version> with the server version. For MySQL use the server version like 5.7.0. For MariaDB use something like mariadb-10.5.9.

Everything except <version> can be found in etc/dbpasswords.secret.

For the judgeadaemon, use the -v commandline option to increase verbosity. It takes a numeric argument corresponding to the syslog loglevels. Use -v 7 to enable loglevel debug. This will also show detailed debugging information from the scripts invoked by the judgedaemon.

A special case is the API user with only the judgedaemon role. For this user, Symfony profiling is disabled on the API for performance reasons even in dev mode. If you should wish to profile these API calls specifically, change webapp/src/EventListener/ProfilerDisableListener.php to enable it.

Running the test suite

The DOMjudge sources ship with a comprehensive test-suite that contains unit, integration and functional tests to make sure the system works.

These tests live in the webapp/tests directory.

To run them, follow the following steps:

  • Make sure you have a working DOMjudge installation.

  • Create a new database with the same name as your normal database, but then postfixed with _test. Make sure your database user has the same permissions on it as the normal database.

  • Make sure your test database contains only the sample data. This can be done by first dropping any existing database and then running APP_ENV=test bin/dj_setup_database -u root -r install.

Note that you don’t have to drop and recreate the database every time you run the tests; the tests are written in such a way that they keep working, even if you run them multiple times.

The file webapp/.env.test (and webapp/.env.test.local if it exists) are loaded when you run the unit tests. You can thus place any test-specific settings in there.

Now to run the tests, execute the command:

webapp/bin/phpunit -c webapp/phpunit.xml.dist

This command can take an argument --filter to which you can pass a string which will be used to filter which tests to run. For example, to run only the jury print controller tests, run:

webapp/bin/phpunit -c webapp/phpunit.xml.dist --filter \
  'App\\Tests\\Controller\\Jury\\PrintControllerTest'

Or to run only one test in that class, you can run:

webapp/bin/phpunit -c webapp/phpunit.xml.dist --filter \
  'App\\Tests\\Controller\\Jury\\PrintControllerTest::testPrintingDisabledJuryIndexPage

Note that most IDEs have support for running tests inside of them, so you don’t have to type these filters manually. If you use such an IDE, just make sure to specify the webapp/phpunit.xml.dist file as a PHPUnit configuration file and it should work.

Loading development fixture data

To debug failing Unit tests the fixtures can be loaded with: ./webapp/bin/console domjudge:load-development-data SampleSubmissionsFixture in the current database.

Installing, updating & upgrading dependencies

To manage the PHP dependencies, use the commands below from the webapp directory.

To require a new dependency/library: composer require package/name or composer require package/name:version, this is most often needed when a newer version is broken, or when you need a specific new feature or new library.

To update all dependencies: composer update, this is useful before a new contest when there is enough time to fix possible issues.

To update a single dependency: composer update package/name and composer update -W package/name for either a single package upgrade or also the other requirements if needed. In the second case, composer is allowed to update a dependency in case the dependency package/name requires a higher version.

When you expect a dependency to upgrade and it doesn’t, you can check on Packagist if a newer version exists and change the version in composer.json, or use composer require package/name:version and rerun the commands above.

Additional developer documentation