2. Installation

2.1. Download

Get the library from the PyPi repository with the pip command, preferably using the --user option:

pip install --user mqttgateway

Alternatively use the bare pip command if you have administrator rights or if you are in a virtual environment.

pip install mqttgateway

Running pip also installs an executable file (exe in Windows or executable python script in Linux) called dummy2mqtt. It launches the demo interface dummy with the default configuration. Its location should be %APPDATA%\Python\Scripts\dummy2mqtt.exe on Windows and ~/.local/bin/dummy2mqtt on Linux (it probably depends on the distribution though…). If not, please search for the file manually.

Also, those same locations should be already defined in the PATH environment variable and therefore the executable should launch from any working directory. If not, the variable will have to be updated manually, or the executable needs to be specified with its real path.

2.2. Configuration

A configuration file is needed for each interface. In the library, the default interface dummy has its own configuration file dummy2mqtt.conf inside the package folder.

The configuration file has a standard INI syntax, as used by the standard library ConfigParser with sections identified by [SECTION] and options within sections identified by option:value. Comments are identified with a starting character #.

There are four sections:

  1. [MQTT] defines the MQTT parameters, most importantly the IP address of the broker under the option host. The address of the MQTT broker should be provided in the same format as expected by the paho.mqtt library, usually a raw IP address (192.168.1.55 for example) or an address like test.mosquitto.org. The default port is 1883, if it is different it can also be indicated in the configuration file under the option port. Authentication is not available at this stage.
  2. [LOG] defines the different logging options. The library can log to the console, to a file, send emails or just send the logs to the standard error output. By default it logs to the console.
  3. [INTERFACE] is the section reserved to the actual interface using this library. Any number of options can be inserted here and will be made available to the interface code through a dictionary initialised with all the option:value pairs.
  4. [CONFIG] is a section reserved to the library to store information about the configuration loading process. Even if it is not visible in the configuration file it is created at runtime.

For more details about the .conf file, its defaults and the command line arguments, go to Configuration.

2.3. Launch

If pip installed correctly the executable files, just launch it from anywhere:

dummy2mqtt

Launched without argument, the application looks for a configuration file in the same directory as the targeted script with the same name as the application, with a .conf extension. In this case, it finds the file dummy2mqtt.conf inside the package folder:

# Configuration file for dummy2mqtt. -----------------------------------------------------------------------------
[CONFIG]
# Reserved section used by the loader to store the location where
#   the configuration settings are coming from, or to store
#   the error if there was one.

# -----------------------------------------------------------------------------
[INTERFACE]
# Section for whatever options are needed by the gateway interface
#   being developed. All these options will be written in a
#   dictionary and passed to the interface.

# Example (it can be checked in the logs):
port: /dev/tty

# -----------------------------------------------------------------------------
[MQTT]
# The parameters to connect to the MQTT broker
host: test.mosquitto.org
port: 1883
keepalive: 60

# The client id can be provided here; if left empty it defaults to the application name
clientid:

# This is the timeout of the 'loop()' call in the MQTT library
timeout: 0.01

# Mapping option. By default it is off.
mapping: on

# Map file: there needs to be a mapping file if the <mapping> option is on.
#   If the <mapfilename> option is left blank, the mapping option is turned
#   off, whatever the value of the <mapping> option.
#   To use the default name and path, use a dot <.> for this option.
#   The default name used is <*application_name*.map>.
#   See below for other instructions on file names and paths.
mapfilename: dummy_map.json

# The 'root' keyword for all MQTT messages.
#   Only necessary if <mapping> is off, disregarded otherwise
#   as the keyword should then be found in the mapping file.
#   By default it is empty ("").
# Example:
#root: home
# Here we don't need it because we use a map.
root:

# The topics to subscribe to, separated by a comma.
#   Only necessary if <mapping> is off, disregarded otherwise
#   as the topics should then be found in the mapping file.
# Example:
#topics: home/lighting/#, home/+/mygateway/#
# Here we don't need it because we use a map.
topics:

# -----------------------------------------------------------------------------
[LOG]
# Logs: all WARN level logs and above are sent to stderr or equivalent.
#   3 more log outputs can be set up: console, rotating files and email.
# Log levels: indicate what log levels are required for each log output.
#   Levels are indicated with the following strings (from the logging module):
#   CRITICAL, ERROR, WARN or WARNING, INFO and DEBUG; use NONE if unused.

# Console level: these are the logs directed to stdout.  Usually only used for testing.
consolelevel: DEBUG

# Log file: file location if logs to file is required.
#   Leave this option blank to not enable a log file.
#   Use a dot <.> to use the default name and path.
#   The default name used is <*application_name*.log>.
#   Make sure the application will have the rights to write in this file.
#   See below for other instructions on file names and paths.
logfilename:

# File level: level for logs directed to the file named by the <logfilename> option.
#   If that option is blank, there is no file log whatever value is given to the option
#   <filelevel> (there is no default file).
filelevel: NONE

# Number of files required for the rotating files. Default is 3.
#filenum:3

# Maximum size of each log file, in KB. Default is 50'000.
#filesize: 50000

# Email credentials; leave empty if not required.
#   All CRITICAL level logs are sent to this email, if defined.
#   For now there is no authentication.
emailhost:
# for example: emailhost: 127.0.0.1
emailport:
# for example: emailport: 25
emailaddress:
# for example: address: me@example.com

#------------------------------------------------------------------------------
# Note on file paths and names:
#   - the default name is 'application_name' +
#                          default extension (.log, .map, ... etc);
#   - the default directories are (1) the configuration file location, (2) the
#     current working directory, (3) the application directory, which
#     'should' be the location of the launching script;
#   - empty file paths have different meaning depending where they are used;
#     best to avoid;
#   - file paths can be directory only (ends with a '/') and are appended with
#     the default name;
#   - file paths can be absolute or relative; absolute start with a '/' and
#     relative are prepended with the default directory;
#   - file paths can be file only (no '/' whatsoever) and are prepended with
#     the default directory;
#   - use forward slashes '/' in any case, even for Windows systems, it should
#     work;
#   - however for Windows systems, use of the drive letter might be an issue
#     and has not been tested.
#------------------------------------------------------------------------------

With the configuration provided, the application uses test.mosquitto.org as MQTT broker and will log messages from all levels only into the console.

Once started, the application logs a banner message and the full configuration used. Check here that all the options are as intended.

Then the log should show if the MQTT connection was successful and display the topics to which the application has subscribed.

After the start-up phase, the dummy interface logs any MQTT messages it receives. It also emits a unique message every 30 seconds.

Start your a MQTT monitor app (I use mqtt-spy). Connect to your MQTT broker (here it is test.mosquitto.org) and subscribe to the topic:

testmqttgtw/dummyfunction/#

You should see the messages arriving every 30 seconds in the MQTT monitor, as well as in the log.

As the application has subscribed as well to this same topic testmqttgtw/dummyfunction/#, it receives back from the broker the same message it just sent, as can be seen in the log.

Publish now a message from the MQTT monitor:

topic: testmqttgtw/dummyfunction//kitchen//me/C
payload: audio_on

You should see in the log that the MQTT message has been received by the gateway, and that it has also been processed correctly by the mapping processor: a first log indicates that the MQTT message has been received by the mqttgateway library, a second log indicates that the internal message has been received by the dummy interface, with the changed (mapped) values of the various characteristics.

Note

When the application sends a message with a topic it has subscribed to (as above), it receives it back from the broker, as seen before. Indeed a log showed that the MQTT message was received by the library. However, because of a feature that silences echo messages (via the sender characteristic), the library stops the message and does not send it to the dummy interface. That is why there is no second log in that case.

2.4. The mapping data

The mapping data is an optional feature that allows to map some or every keyword in the MQTT vocabulary into the equivalent keyword in the interface. This mapping is a very simple many-to-one relationship between MQTT and internal keywords for each characteristic, and its use is only to isolate the internal code from any changes in the MQTT vocabulary.

For the dummy interface, the mapping data is provided by the text file dummy_map.json. It’s just there as an example and it is enabled in the configuration provided. If you send MQTT messages with MQTT keywords from the mapping file, you should see their translation in the logs.

Note that the map file also contains the root of the MQTT messages and the topics that the interface should subscribe to.

For more details on the mapping data, go to Description.

2.5. Deploying a gateway

The objective of developing a gateway is to ultimately deploy it in a production environment. To install a gateway as a service on a linux machine, go to Configuration.