Guided tour

Foreword

This code has been developed chunk by chunk over more than 10 years now, starting with python2.3 or 4. Now it is a python3.6 software and my python skills have fortunately improved over the years. Problem is that several old parts, even after big efforts to correct the worst pieces, are not very idiomatic, especially the core.

Luckily there are unit tests, and whatever one might think about them, it’s not difficult to admit that they’re extremely useful to check nothing got broken when the core parts are written anew or debugged.

The documentation has been originately written using doxygen. Despite the fact it is an excellent documentation software, I have decided to move to Sphinx because it corresponds closer to python best practices. So, all doxygen-style comments will be turned into docstrings so mathmaker can use Sphinx to build the documentation. At the moment this work is just started, so the auto-generated Sphinx documentation is quite uncomplete now.

So, a part of the work to do is surely to bring new features, but another part, more annoying, is to turn ugly old parts into the right pythonic idioms. That’s why at places you’ll see that this or this other module is deprecated and should be “reduced”, or rewritten. A list of such things to do is available on sourceforge.

The issue

It is utmost important to understand that mathmaker is not a software intended to compute mathematical stuff, but to display it. For instance, resolving a first-degree equation is not in itself a goal of mathmaker, because other softwares do that already (and we don’t even need any software to do it). Instead, mathmaker will determine and display the steps of this resolution. Whenever possible, mathmaker solutions will try to mimic the pupils’ way of doing things.

For instance, it won’t automatically simplify a fraction to make it irreducible in one step, but will try to reproduce the steps that pupils usually need to simplify the fraction. So the GCD is only used to check when the fraction is irreducible and for the cases where there’s no other choice, but not as the mean to simplify a fraction directly (not before pupils learn how to use it, at least).

Another example is the need of mathmaker to control the displaying of decimal and integer numbers perfectly. Of course, most of the time, it doesn’t matter if a computer tells that 5.2×5.2 = 27.040000000000003 or 3.9×3.9 = 15.209999999999999 because everyone knows that the correct results are 27.04 and 15.21 and because the difference is not so important, so in many situations, this precision will be sufficient. But, mathmaker can’t display to pupils that the result of 5.2×5.2 is 27.040000000000003.

Also, the human rules we use to write maths are full of exceptions and odd details we don’t notice usually because we’re familiar to them. We would never write

+2x² + 1x - 1(+5 - 1x)

but instead

2x² + x - (5 - x)

There are many conventions in the human way to write maths and many exceptions.

These are the reasons why the core is quite complex: re-create these writing rules and habits on a computer and let the result be readable by pupils is not an easy thing.

Workflow

Mathmaker creates Sheets of maths Exercises.

Each Exercise contains Questions.

Each Question uses objects from the core, that embbed enough information to compute and write the text of the Question and also the answer.

The main executable (entry_point() in mathmaker/cli.py) performs following steps:

  • Load the default settings from configuration files.
  • Setup the main logger.
  • Check that the correct dependencies are installed.
  • Parse the command-line arguments, updates the settings accordingly.
  • Install the language and setup shared objects, like the database connection.
  • If the main directive is list, it just write the directives list to stdout
  • Otherwise, it checks that the directive matches a known sheet (either a yaml or xml file or a sheet’s name that mathmaker provides) and writes the result to the output (stdout or a file) (xml will be dropped in 0.7.2)

The directories

Directories that are relevant to git, at the root:

.
├── docs
├── mathmaker
├── tests
└── toolbox
  • The usual docs/ and tests/ directories
  • mathmaker/ contains the actual python source code
  • toolbox/ contains several standalone scripts that are useful for developers only (not users)
  • Several usual files (.flake8 etc.)
  • outfiles/ (not listed here, because it is not relevant to git) is where the garbage is put (figures created when testing, etc.). Sometimes it is useful to remove all garbage files it contains.

mathmaker/’s content:

$ tree -d -L 1 mathmaker -I __pycache__
mathmaker
├── data
├── lib
├── locale
└── settings
  • data/ is where the database is stored, but also yaml files containing additional wordings, translations etc.
  • lib/ contains all useful classes and submodules (see below).
  • locale/ contains all translation files.
  • settings/ contains the functions dedicated to setup the settings and also the default settings files themselves.

lib/’s content:

$ tree -d -L 3 mathmaker/lib -I __pycache__
mathmaker/lib
├── constants
├── core
├── document
│   ├── content
│   │   ├── algebra
│   │   ├── calculation
│   │   ├── geometry
│   │   └── ... (maybe some others in the future)
│   └── frames
├── machine
├── old_style_sheet
│   └── exercise
│       └── question
└── tools
  • constants/ contains several constants (but pythagorean.py must be replaced by requests to the database)
  • core/ contains all mathematical objects, numeric or geometric
  • document/ contains the frames for sheets, exercises in questions, under document/frames/, and the questions’ content, under document/content/.
  • machine/ contains the “typewriter”
  • old_style_sheet/ contains all old style sheets, exercices and questions. All of this is obsolete (will be replaced by generic objects that take their data from yaml files and created by the objects defined in document/frames/)
  • tools/ contains collections of useful functions
    • __init__.py contains various functions
    • database.py contains all functions required to interact with mathmaker’s database
    • frameworks.py contains a collection of useful functions to handle the collection of yaml sheet files
    • ignition.py contains several functions called at startup
    • maths.py contains some extra mathematical functions
    • wording.py contains a collection of useful functions to handle wordings
    • xml.py contains a collection of useful functions to handle the xml files (obsolete, will disappear in 0.7.2)
  • shared.py contains objects and variables that need to be shared (except settings), like the database connection

Overview of the main classes

A Machine is like a typewriter: it turns all printable objects (Sheets, and everything they contain) into LaTeX. It knows how to turn a mathematical expression in LaTeX format. It knows how to draw figures from the geometrical objects (using eukleides).

The Sheet objects given to a Machine contain guidelines for the Machine: the layout of the Sheet and what Exercises it contains.

The Exercise objects contain Questions and also layout informations that might be specific to the exercise (for instance, display the equations’ resolutions in two columns).

The Question objects contain the mathematical objects from the core and uses them to compute texts and answers. The real content is in lib/document/content/*/*.py. The appropriate module is used by the Question object (defined in lib/document/frames/question.py) to create the question’s mathematical objects, wording and answer.

The objects from the core are all different kinds of mathematical objects, like Sums, Products, Equations or Triangles, Tables… For instance, a Question about Pythagora’s theorem would embed a RightTriangle (which itself embeds information on its sides, vertices, angles; and enough methods to create a picture of it) but also fields telling if the figure should be drawn in the Question’s text or if only a description of the figure should be given; if the hypotenuse should be calculated or another side; if the result should be a rounded decimal and how precise it should be etc.

When a new Sheet is created, all objects it contains are created randomly, following some rules, though, to avoid completely random uninteresting results.

More details about the core objects a little bit below, in the paragraph about The core.

Start working on mathmaker

Short version

Warning

The work is currently (0.7.1) done with python 3.6.

Install dependencies:

  • Ubuntu:

    $ sudo apt-get install eukleides libxml2-utils gettext texlive-full
    
  • FreeBSD:

    $ sudo pkg install python36 py36-sqlite3 gettext eukleides libxml2 texlive-full
    $ rehash
    

And FreeBSD users should check the eukleides install fix for FreeBSD

To install mathmaker in dev mode in a venv, get to the directory where you want to work, and (assuming git and python3.6 are installed):

  • Ubuntu:

    $ python3 -m venv dev0
    $ source dev0/bin/activate
    (dev0) $ pip3 install pytest tox flake8 pydocstyle sphinx sphinx-autodoc-annotation sphinx-rtd-theme
    (dev0) $ mkdir mathmaker
    (dev0) $ cd mathmaker/
    (dev0) $ git clone https://github.com/nicolashainaux/mathmaker.git
    (dev0) $ python3 setup.py develop
    
  • FreeBSD:

    $ python3 -m venv dev0
    $ source dev0/bin/activate.csh
    [dev0] $ sudo pip3 install pytest tox flake8 pydocstyle sphinx sphinx-autodoc-annotation sphinx-rtd-theme
    [dev0] $ mkdir mathmaker
    [dev0] $ cd mathmaker/
    [dev0] $ git clone https://github.com/nicolashainaux/mathmaker.git
    [dev0] $ python3 setup.py develop
    

Usage: get to an empty directory and:

(dev0) $ mathmaker test_11_2 > out.tex
(dev0) $ lualatex out.tex

You can check out.pdf with the pdf viewer you like.

Run the tools:

(dev0) $ cd path/to/mathmaker/tools/
(dev0) $ ./build_db.py
(dev0) $ ./update_pot_files

Most of the tests are stored under tests/. Some others are doctests. Any new test or doctest will be added automatically to the tests run by py.test or tox.

Run the tests:

(dev0) $ py.test
(dev0) $ tox

Tox will ignore missing python interpreters.

Edit the settings:

(dev0) $ cd path/to/mathmaker/settings/
(dev0) $ mkdir dev/
(dev0) $ cp default/*.yaml dev/

In dev/logging.yaml you can set the __main__ logger to INFO (take care to define log rotation for /var/log/mathmaker). Set the dbg logger to DEBUG.

Each debugging logger can be enabled/disabled individually in debug_conf.yaml (by setting it to DEBUG or INFO).

See Loggers: main, daemon, debugging, output watcher for more details on how to setup new loggers (and debugging loggers).

You can override settings in dev/user_config.yaml to your liking.

Before starting, you should read at least the Auxiliary tools and Writing rules sections. It is certainly worth also to have a look at Advanced features.

Hope you’ll enjoy working on mathmaker!

Detailed version

Dev environment

Note

python3.6 is mandatory for mathmaker development

Install external dependencies

You’ll need to install the same dependencies as users do (see Install). In addition, xgettext is required to extract the gettext messages from py files. In Ubuntu 14.04 it’s in the gettext package.

Get mathmaker’s source code from github repo

In the folder of your choice:

$ git clone https://github.com/nicolashainaux/mathmaker.git

Setup a python virtual environment

It is strongly advised to install mathmaker in develop mode inside of a python virtual environment. This allows to install the required libraries without conflicting with other projects or python software on the same computer. Just get to the directory of your choice, and to create a virtual environment named dev0, you type:

$ python3 -m venv dev0

From there, you can activate it:

on Ubuntu:

$ source dev0/bin/activate

on FreeBSD:

$ source dev0/bin/activate.csh

Install mathmaker

Once your virtual environment is activated, go to mathmaker’s root:

(dev0) $ cd path/to/mathmaker/

You should see something like:

(dev0) $ ls
CHANGELOG.rst  docs  LICENSE  MANIFEST.in  mathmaker README.md  README.rst  requirements.txt  setup.py  tests  tools  tox.ini

There you can install mathmaker in developer mode:

(dev0) $ python3 setup.py develop

It’s possible to clean the project’s main directory:

(dev0) $ python3 setup.py clean

Run mathmaker and tools

From now on, it is possible to run mathmaker from your virtual environment. As mathmaker is installed in developer mode, any change in the source files will be effective when running mathmaker. Go to a directory where you can leave temporary files (each sheet requiring pictures will produce picture files, by default), and test it:

(dev0) $ cd path/to/garbage/directory/
(dev0) $ mathmaker test_11_2 > out.tex
(dev0) $ lualatex out.tex

You can check out.pdf with the pdf viewer you like.

You can also run the tools:

(dev0) $ cd path/to/mathmaker/
(dev0) $ cd toolbox/
(dev0) $ ./build_db.py
(dev0) $ ./update_pot_files

Somewhat below, more informations about the Auxiliary tools.

Once you’re done working with mathmaker, you can deactivate the virtual environment:

(dev0) $ deactivate
$

Note that it is possible to run mathmaker outside the virtual environment this way:

$ cd path/to/mathmaker/
$ python3 -m mathmaker.cli

But it requires to have installed the python dependencies yourself on the host system (e.g. the computer) and maybe also to have set $PYTHONPATH correctly (and exported it).

Other dependencies

Linters

It is recommended to install linters for PEP 8 and PEP 257 (see Writing rules):

(dev0) $ pip3 install flake8
(dev0) $ pip3 install pydocstyle
Test dependencies

In addition you should install at least py.test, and also tox if you intend to run tox tests:

(dev0) $ pip3 install pytest
(dev0) $ pip3 install tox

Below is more information about testing.

Documentation dependencies

You’ll need to install these dependencies in the virtual environment:

(dev0) $ pip3 install sphinx sphinx-rtd-theme

sphinx-rtd-theme is the theme used for mathmaker’s documentation. It’s the readthedocs theme.

Note

sphinx-autodoc-annotation makes writing docstrings lighter when using python3 annotations. Problem is, this package currently has a bug that prevents to build the doc on readthedocs.

Below is more information about documentation.

Dev settings

You can make a copy of the default configuration files:

(dev0) $ cd path/to/mathmaker/
(dev0) $ cd settings/
(dev0) $ mkdir dev/
(dev0) $ cp default/*.yaml dev/

Then you can edit the files in mathmaker/settings/dev/ to your liking. Any value redefined there will override all other settings (except the options from the command line).

In logging.yaml the loggers part is interesting. I usually set the __main__ logger to INFO (this way, informations about starting and stopping mathmaker are recorded to /var/log/mathmaker, take care to define the log rotation if you do so) and the dbg logger to DEBUG. This second setting is important because it will allow to enable debugging loggers in debug_conf.yaml.

debug_conf.yaml allows to trigger each debugging logger individually by setting it to DEBUG instead of INFO.

And in user_config.yaml it is especially nice to define an output directory where all garbage files will be stored, but also to set the language, the font etc.

For instance, my settings/dev/user_config.yaml contains this:

# SOFTWARE'S CONFIGURATION FILE

PATHS:
    OUTPUT_DIR: /home/nico/dev/mathmaker/poubelle/

LOCALES:
    LANGUAGE: fr_FR
    CURRENCY: euro

LATEX:
    FONT: Ubuntu
    ROUND_LETTERS_IN_MATH_EXPR: True

DOCUMENT:
    QUESTION_NUMBERING_TEMPLATE_SLIDESHOWS: "n°{n}"

See Settings to learn more about the way settings are handled by mathmaker.

Testing

Run the tests

The testing suite is run by py.test this way:

(dev0) $ py.test

or this way:

(dev0) $ python3 setup.py test

Where do they live?

Most of the tests belong to tests/. Any function whose name starts with test_ written in any python file whose name also starts with test_ (and stored somewhere under tests/) and will be automatically added to the tests run by py.test.

Some more tests are written as doctests (see also pytest documentation about doctests) in the docstrings of the functions. It’s possible to add doctests, especially for simple functions (sometimes it is redundant with the tests from tests/, but this is not a serious problem). The configuration for tests is so that any new doctest will be automatically added to the tests run by py.test.

Tox

To test mathmaker against different versions of python, you can run tox this way:

(dev0) $ tox

or this way:

(dev0) $ python3 setup.py tox

Be sure you have different versions of python installed correctly on your computer before starting this. The missing versions will be skipped anyway. Note that it is not a purpose of mathmaker to run under a lot of python versions (several python3 versions are OK, but no support for python2 is planned, unless someone really wants to do that).

Loggers: main, daemon, debugging, output watcher

See Dev settings to know how to use the settings files and enable or disable logging and debugging.

Main logger

__main__ is intended to be used for messages relating to mathmaker general working. In particular, it should be used to log any error that forces mathmaker to stop, before it stops.

In order to use this __main__ logger, you can write this at the start of any function (assuming you have imported settings at the top of the file):

log = settings.mainlogger

And then inside this function:

log.error("message")

(or log.warning("message") or log.critical("message") depending on the severity level).

If an Exception led to stop mathmaker, then the message should include its Traceback (if you notice this is not the case somewhere, you can modify this and make a pull request). For instance in cli.py:

try:
    shared.machine.write_out(str(sh))
except Exception:
    log.error("An exception occured during the creation of the sheet.",
              exc_info=True)
    shared.db.close()
    sys.exit(1)

Daemon logger

This logger is intended to be used by the daemon script. Works the same way as the main logger.

Debugging logger

dbg is the logger dedicated to debugging and ready to use. No need to write sys.stderr.write(msg) anywhere.

If there’s no logger object in the function you want to print debugging messages, you can create one this way:

  • Add the matching entry in debug_conf.yaml (both the settings/default/ and settings/dev/ versions, but set to INFO in the settings/default/ version). For short modules, you can add only one level, and for modules containing lots of functions of classes, two levels should be added, like the example of the extract below:

    dbg:
        db: INFO
        wording:
            merge_nb_unit_pairs: INFO
            setup_wording_format_of: INFO
            insert_nonbreaking_spaces: INFO
        class_or_module_name:
            fct: DEBUG
    
  • Import the settings at the top of the file, if it’s not done yet:

    from mathmaker import settings
    
  • Create the logger at the start of the function (i.e. locally):

    def fct():
        log = settings.dbg_logger.getChild('class_or_module_name.fct')
    
  • Then where you need it, inside fct, write messages this way:

    log.debug("the message you like")
    

Later when you need to disable this logger, you just set it to INFO instead of DEBUG in settings/dev/debug_conf.yaml. See Dev settings for information on these files.

A summary of the conventions used to represent the different core objects (i.e. what their __repr__() returns):

_images/dbg_all.png

Output Watcher logger

This is another debugging logger. It can be used to check wether output is as expected, in order to detect bugs that do not crash mathmaker. Works the same way as the main logger. The log messages are sent to another facility (local4), in order to be recorded independently.

System log configuration

Systems using rsyslog

The communication with rsyslog goes through a local Unix socket (no need to load rsyslog TCP or UDP modules).

Note

The default socket is /dev/log for Linux systems, and /var/run/log for FreeBSD. These values are defined in the logging*.yaml settings files.

rsyslog may be already enabled and running by default (Ubuntu) or you can install, enable and start it (in Manjaro, # systemctl enable rsyslog and # systemctl start rsyslog).

Ensure /etc/rsyslog.conf contains these lines:

$ModLoad imuxsock

$IncludeConfig /etc/rsyslog.d/*.conf

Then create (if not created yet) a ‘local’ configuration file, like: /etc/rsyslog.d/40-local.conf and put (or add) in it:

#  Local user rules for rsyslog.
#
#
local4.*                     /var/log/mathmaker_output.log
local5.*                     /var/log/mathmaker.log
local6.*                     /var/log/mathmakerd.log

Then save it and restart:

  • in Ubuntu: # service rsyslog restart
  • in Manjaro: # systemctl restart rsyslog

Warning

Do not create /var/log/mathmaker.log yourself with the wrong rights, otherwise nothing will be logged.

To format the messages in a nicer way, it’s possible to add this in /etc/rsyslog.conf:

$template MathmakerTpl,"%$now% %timegenerated:12:23:date-rfc3339% %syslogtag%%msg%\n"

and then, modify /etc/rsyslog.d/40-local.conf like:

local4.*                        /var/log/mathmaker_output.log;MathmakerTpl
local5.*                        /var/log/mathmaker.log;MathmakerTpl
local6.*                        /var/log/mathmakerd.log;MathmakerTpl

Tools to check everything’s fine: after having restarted rsyslog, enable some more informations output:

# export RSYSLOG_DEBUGLOG="/var/log/myrsyslogd.log"
# export RSYSLOG_DEBUG="Debug"

and running the configuration validation:

# rsyslogd -N2 | grep "mathmaker"

should show something like (errorless):

rsyslogd: version 7.4.4, config validation run (level 2), master config /etc/rsyslog.conf
2564.153590773:7f559632b780:   ACTION 0x2123160 [builtin:omfile:/var/log/mathmaker.log;MathmakerTpl]
2564.154126386:7f559632b780:   ACTION 0x2123990 [builtin:omfile:/var/log/mathmakerd.log;MathmakerTpl]
2564.158461309:7f559632b780:   ACTION 0x2123160 [builtin:omfile:/var/log/mathmaker.log;MathmakerTpl]
2564.158729012:7f559632b780:   ACTION 0x2123990 [builtin:omfile:/var/log/mathmakerd.log;MathmakerTpl]
rsyslogd: End of config validation run. Bye.

Once you’ve checked this works as expected, do not forget to configure your log rotation.

Note

mathmaker does not support systemd journalisation (the default one in Manjaro). You may have to setup systemd too (enable ForwardToSyslog in its conf file) in order to get rsyslog recording messages. Also you may need to add $ModLoad imjournal in /etc/rsyslog.conf and to create the file /var/spool/rsyslog. For a better setup, see https://www.freedesktop.org/wiki/Software/systemd/syslog/. A workaround to prevent duplicate messages could be to discard the unwanted ones, like described here: http://www.rsyslog.com/discarding-unwanted-messages/.

Documentation

Current state

As stated in the Foreword, the documentation is being turned from doxygen to Sphinx, so there are missing parts .

Any new function or module has to be documented as described in PEP 257.

The doxygen documentation for version 0.6 is here. The core parts are still correct, so far.

Format

This documentation is written in ReStructured Text format.

There are no newlines inside paragraphs. Set your editor to wrap lines automatically to your liking.

Make html

To produce the html documentation:

(dev0) mathmaker [dev] $ $ cd docs/
(dev0) mathmaker/docs [dev] $ $ make html

If modules have changed (new ones, deleted ones), it is necessary to rebuild the autogenerated index:

(dev0) mathmaker/docs [dev] $ sphinx-apidoc -f -o . ../mathmaker

Auxiliary tools

Several standalone scripts live in the toolbox/ directory under root. They can be useful for several tasks that automate the handling of data.

The two most useful ones are both meant to be run from the toolbox/ directory. They are:

  • build_db.py, used to update the database when there are new entries to add in it. If new words of 4 letters are added to any po file, build_db.py should be run, it will add them to the database. If new wordings are entered in mathmaker/data/wordings/*.xml (obsolete: xml files will be replaced by yaml files up from 0.7.2), then it should be run too. See details in the docstring. And if a new table is required, it should be added in this script. For instance, the pythagorean triples should live in the database and will be added to this list soon or later.
  • update_pot_files, a shell script making use of xgettext and of the scripts merge_py_updates_to_main_pot_file, merge_yaml_updates_to_pot_file and merge_xml_updates_to_pot_file (this last one will be removed in 0.7.2). Run update_pot_files to update locale/mathmaker.pot when new strings to translate have been added to python code (i.e. inside a call to _()) or new entries have been added to any yaml or xml (xml files will be turned to yaml files in 0.7.2) file from mathmaker/data (only entries matching a number of identifiers are taken into account, see DEFAULT_KEYWORDS in the source code to know which ones exactly).
  • build_index.py will build the index of available sheets. Run it when you need to test a new sheet.

import_msgstr and retrieve_po_entries are useful on some rare occasions. See their docstrings for more explanations. They both have a --help option.

pythagorean_triples_generator shouldn’t be of any use any more (later on maybe a part of its code will be incorporated to build_db.py, that’s why it’s still around here)

Writing rules

It is necessary to write the cleanest code possible. It has not been the case in the past, but the old code is updated chunk by chunk and any new code portion must follow python’s best practices, to avoid adding to the mess, and so, must:

And of course, all the code is written in english.

As to PEP 8, mathmaker ‘s code being free from errors, the best is to use a linter, like flake8. They also exist as plugins to various text editors or IDE (see Atom packages for instance). Three error codes are ignored (see .flake8):

  • E129 because it is triggered anytime a comment is used to separate a multiline conditional of an if statement from its nested suite. A choice has been made to wrap multiline conditions in () and realize the separation with next indented block using a # __ comment (or any other comment if it’s necessary) and this complies with PEP 8 (second option here):

    Acceptable options in this situation include, but are not limited to:

    # No extra indentation.
    if (this_is_one_thing and
        that_is_another_thing):
        do_something()
    
    # Add a comment, which will provide some distinction in editors
    # supporting syntax highlighting.
    if (this_is_one_thing and
        that_is_another_thing):
        # Since both conditions are true, we can frobnicate.
        do_something()
    
  • W503 because PEP 8 does not compel to break before binary operators (the choice of breaking after binary operators has been done).

  • E704 because on some occasions it is OK to put several short statements on one line in the case of def. It is the case in several test files using lines like def v0(): return Value(4)

Other choices are:

  • A maximum line length of 79
  • Declare _ as builtin, otherwise all calls to _() (i.e. the translation function installed by gettext) would trigger flake8’s error F821 (undefined name).
  • No complexity check. This might change in the future, but the algorithms in the core are complex. It’s not easy to make them more simple (if anyone wants to try, (s)he’s welcome).
  • Name modules, functions, instances, and other variables in lower case, whenever possible using a single word but if necessary, using several_words_separated_with_underscores.
  • Name classes in CapitalizedWords, like: SuchAWonderfullClass (don’t use mixedCase, like wrongCapitalizedClass).
  • All import statements must be at the top of any module. It must be avoided to add from ... import ... at the top of some functions, but sometimes it’s necessary. A solution to avoid this is always preferred.
  • All text files (including program code) are encoded in UTF-8.

As to PEP 257, this is also a good idea to use a linter, but lots of documentation being written as doxygen comments, the linter will detect a lot of missing docstrings. Just be sure the part you intend to push does not introduce new PEP 257 errors (their number must decrease with time, never increase).

The text of any docstring is marked up with reStructuredText.

The module mathmaker.lib.tools.wording can be considered as a reference on how to write correct docstrings. As an example, the code of two functions is reproduced here.

Note

The use of python3’s annotations and sphinx-autodoc-annotation would automatically add the types (including return type) to the generated documentation. If sphinx-autodoc-annotation’s bug is corrected, the :type ...: ... and :rtype: ... lines will be removed.

def cut_off_hint_from(sentence: str) -> tuple:
    """
    Return the sentence and the possible hint separated.

    Only one hint will be taken into account.

    :param sentence: the sentence to inspect
    :type sentence: str
    :rtype: tuple

    :Examples:

    >>> cut_off_hint_from("This sentence has no hint.")
    ('This sentence has no hint.', '')
    >>> cut_off_hint_from("This sentence has a hint: |hint:length_unit|")
    ('This sentence has a hint:', 'length_unit')
    >>> cut_off_hint_from("Malformed hint:|hint:length_unit|")
    ('Malformed hint:|hint:length_unit|', '')
    >>> cut_off_hint_from("Malformed hint: |hint0:length_unit|")
    ('Malformed hint: |hint0:length_unit|', '')
    >>> cut_off_hint_from("Two hints: |hint:unit| |hint:something_else|")
    ('Two hints: |hint:unit|', 'something_else')
    """
    last_word = sentence.split()[-1:][0]
    hint_block = ""
    if (is_wrapped(last_word, braces='||')
        and last_word[1:-1].startswith('hint:')):
        # __
        hint_block = last_word
    if len(hint_block):
        new_s = " ".join(w for w in sentence.split() if w != hint_block)
        hint = hint_block[1:-1].split(sep=':')[1]
        return (new_s, hint)
    else:
        return (sentence, "")


def merge_nb_unit_pairs(arg: object):
    r"""
    Merge all occurences of {nbN} {\*_unit} in arg.wording into {nbN\_\*_unit}.

    In the same time, the matching attribute arg.nbN\_\*_unit is set with
    Value(nbN, unit=Unit(arg.\*_unit)).into_str(display_SI_unit=True)
    (the possible exponent is taken into account too).

    :param arg: the object whose attribute wording will be processed. It must
      have a wording attribute as well as nbN and \*_unit attributes.
    :type arg: object
    :rtype: None

    :Example:

    >>> class Object(object): pass
    ...
    >>> arg = Object()
    >>> arg.wording = 'I have {nb1} {capacity_unit} of water.'
    >>> arg.nb1 = 2
    >>> arg.capacity_unit = 'L'
    >>> merge_nb_unit_pairs(arg)
    >>> arg.wording
    'I have {nb1_capacity_unit} of water.'
    >>> arg.nb1_capacity_unit
    '\\SI{2}{L}'
    """

Atom packages

This paragraph lists useful packages for atom users (visit the links to have full install and setup informations):

  • flake8 linter provider: linter-flake8 (Note: you should let the settings as is, except for the “Project config file” entry where you can write “.flake8” to use mathmaker project’s settings.)
  • pydocstyle linter provider: linter-pydocstyle
  • python3’s highlighter: MagicPython (MagicPython is able to highlight correctly python3’s annotations. You’ll have to disable the language-python core package.)
  • To edit rst documentation: language-restructuredtext and rst-preview-pandoc

A deeper look in the source code

Settings

Everything happens in mathmaker/settings/__init__.py (it would be better to have everything happening rather in something like mathmaker/settings/settings.py, so this will most certainly change).

This module is imported by the main script at start, that run its init() function. After that, any subsequent from mathmaker import settings statement will make settings.* available.

The values shared as settings.* are: the paths to different subdirectories of the project, the loggers and the values read from configuration files. (Plus at the moment, two default values that should move to some other place).

None of these values is meant to be changed after it has been set by the main script, what calls settings.init() and then corrects some of them depending on the command-line options. Once this is done, these values can be considered actually as constants (they are not really constants as they are setup and corrected, so no UPPERCASE naming).

tests/conftest.py `` uses the ``settings module the same way mathmaker/cli.py does.

Configuration

load_config() handles this and is defined in mathmaker/lib/tools/__init__.py. It works the same way for any of the *.yaml files. It first loads the default values from mathmaker/settings/default/filename.yaml. Then it updates any value found redefined in any of these files: /etc/mathmaker/filename.yaml, ~/.config/mathmaker/filename.yaml and mathmaker/settings/dev/filename.yaml. Any missing file is skipped (except the first one: the default settings are part of the code, are shipped with it and must be present).

An extended dict class is used to deal easier with dicts created from yaml files. See in mathmaker/lib/tools/__init__.py.

The daemon

It’s a daemonized web server that allows to communicate with mathmaker through http requests. See http server (mathmakerd).

Shared

Three resources are shared: the database, the LaTeX machine and the sources.

mathmaker/lib/shared.py works a similar way as the settings module. It is initialized once in the main script and then its resources are used.

The database

The aim of the database is to avoid having to create a lot of randomly values and store them in lists or dicts everytime we need something.

It is considered as a source among others.

The sources

They concern as well numbers as words or letters or anything one can think of.

Note

Old style sheets don’t use sources.

When random numbers are required, most of the time, we don’t need complete random. For instance if we want a pair of integers for the multiplication tables between 2 and 9, we don’t want to ask the same question twice in a row.

The sources manage this randomness. Anytime we need to use a source, we can use its next() method to get the next random data, without worrying in the same time whether it’s the same as the previous one or not.

So we have sources for names, for words having a limited number of letters, for different kinds of numbers but also for mini-problems wordings.

So far, there are two kinds of sources: the ones that are provided by the database, and the ones that are provided by the function generate_values(). They both reside in mathmaker/lib/tools/database.py.

All sources are initialized in mathmaker/lib/shared.py. There you can see which one has its values provided by the database, which are the other ones.

The database provides an easy way to ensure the next value will be different from the last one: we simply “stamp” each drawn value and the next time we draw a value among the yet unstamped ones. When they’re all stamped, we reset all stamps and redraw. There’s a tiny possibility to draw two times in a row the same value, so far, but it’s so tiny we can safely ignore it. (This could be improved later). The values drawn from generate_values() are so different the ones from the others that it’s very unlikely to draw the same ones two times in a row.

The real and the fake translation files

mathmaker/locale/mathmaker.pot is a real translation file.

The other mathmaker/locale/*.pot files are “fake” ones. They are used to get random words in a specific language, but the words do not need to be the exact translation from a source language.

For instance, w4l.pot contains words of four different letters. It wouldn’t make sense to translate the english word “BEAN” into a word of the same meaning AND having exactly four different letters, in another language. This wouldn’t work for french, for instance. In general this would only work for rare exceptions (like “HALF” can be translated to “DEMI” in french).

The same applies to feminine_names.pot and masculine_names.pot. These files are used to get random names, but we don’t need to translate them.

So, the entries in these “fake” translation files are only labeled entries, with nothing to translate.

A translator only needs to provide a number of entries (at least 10) in each of these files. No matter how many, no matter which msgid do they match. So: in masculine_names.po are several masculine names required, in feminine_names.po are several feminine names required and in w4l.po are several words of four unique letters required. Each time, at least 10, and then, the more the better.

The sheets, exercises and questions

There is still a bunch of “old-style” written sheets, that were not generated from yaml documents. I won’t describe them thoroughly. They will disappear in the future, when they’re replaced by their yaml counterparts. They are kept in lib/old_style_sheet/, so far. They use the classes S_Structure and S_Generic. S_Structure handles the layout of the sheet depending on the SHEET_LAYOUT dict you can find at the top of any sheet module.

Another bunch have been written in XML. They will disappear. So far, mathmaker can read the sheets data from a xml or a yaml document, but the xml format will be dropped in 0.7.2, so don’t bother with it now.

So, all new sheets are stored in yaml files (under data/frameworks/theme/subtheme.yaml, for instance data/frameworks/algebra/expand.yaml).

They are handled by sheet.py, exercise.py and question.py in lib/document/frames/.

The core

Diagram

You can check the 0.6 version (i.e. from doxygen) of the top of the core diagram, though it will be somewhat changed later, it still can be used as reference for some time.

Unfinished draft of future plans:

_images/new_inheritance_2015.png

Core objects’ summary

Objects at left; associated __repr() at right:

_images/all_pics.png

Core objects’ details

The “old” doc for 0.6 version is available here and mainly still correct for 0.7 version. When things will have settled down to something more stable, an updated documentation will be published chunk by chunk.

What can be done?

See the tickets on sourceforge and especially the ones for the 1.0 version.

mathmaker package

Subpackages

Submodules

mathmaker.cli module

mathmaker.cli.entry_point()[source]

mathmaker.daemon module

class mathmaker.daemon.MathmakerHTTPRequestHandler(request, client_address, server)[source]

Bases: http.server.BaseHTTPRequestHandler

do_GET()[source]
mathmaker.daemon.run()[source]

Module contents