Python socket modules

Python socket modules are Python modules where the associated Python script use UDP or TCP sockets to communicate with the framework.

Overview

These modules allow to execute Python code in any Python 2 or Python 3 environment:

• A Python module creates a Python process using the Python executable path defined in the framework properties. By default the framework will look for a Python 2.x executable among the applications installed on the System
• services reception from the Python module will be deferred to the associated Python script using an UDP or TCP communication
• services invocation from the Python script will be deferred as a Python module Service invocation using an UDP or TCP communication

Two generic additional Python scripts are necessary to handle the communication between the Python module and the Python script:

• The pythonModule.py script is responsible for the Services subscription in the Python environment
• The pythonUtils.py script is an utility script usable by the user Python script

Note that these two modules are valid for both a Python 2 or Python 3 environment for UDP communication.

Initialization and runtime sequence

The initialization and runtime sequence of Python scripts follows two general phases:

• The initialization phase: this phase takes place during the module initialization
• The runtime phase: this phase takes place after the module start

Note that if the initialization phase does not end correctly^[1]

It can be the case if the Python script takes too much time to start-up, or if there is an exception during the initialization of the script

, then the Python module initialization will be aborted. Else the runtime phase will be started.

Using lengthy methods for the Python scripts

Python framework

The Python script is not executed in the JVM, but the framework will look for a Python executable on the Platform, or the user can specify the path of the Python executable.

Deployment

These modules do not need a deployment in their parent application because they are scripted.

Declaration

These modules are declared by the top-level pythonModule element. For example:

      <pythonModule name="FlightManagementSystem">

By default these modules use a UDP communication, but it is possible to specify if the communication uses TCP or UDP using the protocol attribute in the pythonImplementation element:

• The udp value specifies that the communication uses UDP
• The tcp or tcpBlocking value specifie that the communication uses TCP. See Python TCP modules for more information

For example, in the following example the communcation uses TCP:

      <pythonModule name="PublishModule" >
         <pythonImplementation path="pythonAppli" protocol="tcp">
            <defaultSendEntryPoint method="publish" />
         </pythonImplementation>
      </pythonModule>

Types limitations

• Python UDP modules have two ways of exchanging data between Java and Python depending on the Python executable version. See Python UDP modules for more information
• Python TCP modules only use Python 3. As there is a JSON library included in the Python implementation, so the exchange of Data is performed using the JSON format. There are no limitations on the types which can be exchanged with Python modules, including types complex types

Python socket module implementation

The pythonImplementation element declares the Python script file which implements the script, and specifies the associated properties.

The following list specifies the properties which are common to Python UDP modules and Python TCP modules:

• path: refer to the name of the file (without the ".py" extension) which implements the Python script. The path of this file is by default relative to the application.xml file which declares the module
• inputPort, inputSize, outputPort, and outputSize: define the ports which will be used to communicate between the java application and the Python Script^[2]
  Note that these ports should not be identical to existing ports used by the framework for the communication between modules
  . All these parameters are optional. If the ports are not specified, the framework will find free local ports. See input and output size. If a size is not specified, the framework will use the Python default port size by default
• waitAtStart: the maximum duration to wait for after the start of the Python script^[3]
  This is necessary to allow the Python executable to start-up, see also Python initialization and runtime sequence
  . By default the duration will be specified as 200 ms if not defined. See also Waiting for Python runtime initialization
• waitAtInit: the optional duration to wait for after calling the Python init EntryPoint. By default the Java code will be specified as 500 ms if not defined. . See also Waiting for Python script initialization
• wait: the optional duration to wait after each Java - Python dialog. By default the duration will be specified as 10 ms if not defined
• pythonEnumAsString: an optional boolean to specify that enumeration values should be sent as names to the Python module rather than their index. See Enumerations as String for more information
• className: the optional name of the Python class to call. By default the name is derived from the name of the Python module. See Python class name for more information
• protocol: specifies the protocol (default is "udp"). Allows to use TCP modules rather than UDP. See declaration for more information
• host: specifies the host. By default if not specified the host will be the localhost (127.0.0.1), but it is possible to specify the value of the host explicitly. See also host for more information
• compatibility: Allows to set a compatibility for an older version of the Python modules. See Python compatibility for more information

Host

The host parameter for the pythonImplementation element allows to specify the host for the Java to/from Python communication. By default if not specified the localhost (127.0.0.1) will be used.

Note that in all the cases the communication must be performed i the same mahcine, so it will also use the lcoalhost, even if this host is set explicitly. However setting the value of the local host explicitly can be useful if you want to use WireShark to debug the UDP or TCP communication between the Java and the Python processes.

Services subscription queue

The pythonQueue child element allows to specify how the services notifications are handled at the python level. This element has the following attributes (all optional):

• queueSize: the optional size of the queue (used for all services notifications by default)
• publishQueueSize: the optional size of the queue for publish Service notifications
• eventQueueSize: the optional size of the queue for event and request services notifications. Note that the event queue has priority on the publish queue

If a queue is full, then all further notifications will be discarded for this queue until the queue has again available spaces.

For example:

        <pythonImplementation path="pythonAppli" inputSize="1024" outputSize="1024">
            <pythonQueue publishQueueSize="20" eventQueueSize="10" />
        </pythonImplementation>

Examples

For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="5000"/>
      </pythonModule>

Another example, where available free local ports are used for inputPort and outputPort, and 1024 is used for both inputSize and outputSize:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" />
      </pythonModule>

Input and output size

inputSize represents the size of the buffer from Java to Python. outputSize represents the size of the buffer from Python to Java:

• If you does not set the value for a size, the Python default port size will be used
• If you set a value which is less than 200, a value of 200 will be used

You may encounter problems if one of the sizes is too small. For example, you will encounter a "Python output Buffer size too small" error if the outputSize is too small.

Values separator

The content of Services sent or received by the Python script is a String containing the service name followed by the values separated by the separator^[4]

The order of the values is the order of the Service declaration

.

Note that this is only useful for Python version up to 2.5. later Python versions use JSON. later Python versions use JSON for the Java / Python communication and don't need a specific separator.

For example for the following Service:

      <services>
         <publish name="position" >
            <data name="latitude" type="float" />
            <data name="longitude" type="float" />
            <data name="altitude" type="float" />
         </publish>
      </services>

We may have the following content exchanged between Java and Python:

      position=latitude:1.23^longitude:0.45^altitude:1200

It is possible to change the default value of the separator to be any character (or even a string having more than one character), including unicode character. This may be useful if string data sent or received by the Python script contain the default separator character.

For example the following declarations are valid:

• <pythonImplementation ... separator="!" >
• <pythonImplementation ... separator="!!" >
• <pythonImplementation ... separator="&#x00B0;" >

EntryPoints

Init EntryPoint

If the init entry point is defined, the module class must thave one method with the following signature:

      def method_name(self, pythonUtils):

with method_name being the name defined for the Start entry point. The method will be called once after the framework has been started.

For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024">
              <initEntryPoint method="init" />
         </pythonImplementation>
      </pythonModule>

Start EntryPoint

If the start entry point is defined, the module class must thave one method with the following signature:

      def method_name(self, pythonUtils):

with method_name being the name defined for the Start entry point. The method will be called once after the framework has been started.

For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024">
              <startEntryPoint method="start" />
         </pythonImplementation>
      </pythonModule>

Default receive entryPoints

The defaultReceiveEntryPoint element defines the default method which will be called by default when being notified by a Service.

For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024">
              <defaultReceiveEntryPoint method="receive" />
         </pythonImplementation>
         <interfaces>
            <push service="service1" />
            <subscribe service="service2" />
         </interfaces>
      </pythonModule>

The method must have the following signature:

      def method_name(self, pythonUtils, serviceName):

Default send EntryPoint

The defaultSendEntryPoint element defines the default method which will be called by default when invoking automatically a publish Service^[5]

some publish Services can be called cyclically

The signature of this method will always be: <send>(ServiceInstance service). The method will be called each time a Service invoked automatically is invoked (just before sending its result).

For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024">
              <defaultSendEntryPoint method="send" />
         </pythonImplementation>
         <interfaces>
            <push service="service1" />
            <subscribe service="service2" />
         </interfaces>
      </pythonModule>

It can be used to perform computations before invoking the Service (for example, setting values to datas).

The method must have the following signature:

      def method_name(self, pythonUtils, serviceName):

Specific entryPoints

It is possible to define a specific Python method name (entry point) for a Service. For example:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024">
              <defaultReceiveEntryPoint method="receive" />
              <defaultSendEntryPoint method="invoke" />
         </pythonImplementation>
         <interfaces>
            <push service="service1" >
               <entryPoint method="subscribe1" />
            </push>
            <subscribe service="service2" >
               <entryPoint method="subscribe2" />
            </subscribe>
         </interfaces>
      </pythonModule>

Python class name

By default the module must contain at least one class which must have the same name as the module file itself^[6]

Minus the first character which must be upper case

. For example, if the file name is myPythonAppli.py, then the path parameter must be myPythonAppli, and the class defined in the module must be named MyPythonAppli.

The module is free to contain other classes as well.

For example:

      <pythonModule name="FlightManagementSystem" id="1" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024"/>
      </pythonModule>

and:

      from pythonUtils import PythonUtils

      class PythonAppli:

However, it is possible to specify the name of the class to call in the module by using the className parameter.

For example:

      <pythonModule name="FlightManagementSystem" id="1" >
         <pythonImplementation path="pythonAppli" className="FMS" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024"/>
      </pythonModule>

and:

      from pythonUtils import PythonUtils

      class FMS:

Enumerations as String

By default, enumeration values for the JSON content sent to the Python module are sent wityh the state value (an int).

The pythonEnumAsString property specifies that the state names should be sent instead. Note that this can also be configured with the framework property of the same name.

Python module implementation constraint

For the moment, there is one limitation on the way the module should be implemented:

• The module must import the PythonUtils class in the pythonUtils module

Python compatibility

The pythonModule.py and pythonUtils.py scripts must change when the communication between Java and Python is upgraded.

The Python modules version is provided by the Framework as a launch argument when starting the Python process. The pythonModule.py script check if its own version is compatible with the version proviced by Java. This means that you normally should replace the pythonModule.py and pythonUtils.py scripts if the Python modules version changes.

Python library

In order for the Python script to work properly, you must copy two additional Python scripts which will be used as a library in the same directory as the script itself:

• The pythonModule.py script
• The pythonUtils.py script

These two supporting scripts do not depend on the content of the Python script and can be retrieved in the Browser by performing the command Tools => Generate Python Library.

Note that you must use the correct version of the Python scripts depending on the version of Python you intend to use. See generating a Python library more information.

PythonUtils.py script

The pythonUtils.py script utility has the following utility methods:

• def getService(self, name):: get a Service with its name
• def setValue(self, service, name, value):: set a Data value for a Service
• def send(self, serviceName):: invoke a Service
• def echo(self, content):: print the value of an element^[7]
  The general case would be a message, but any Python object can be printed
  on the system.out Stream^[8]
  Note that due to the rate of exhcange between Java and Python, this method force a flush. Using only print(content) would not print the content in many cases
• def err(self, message):: print a message on the system.err Stream

Note that to get a Data value for a Service, you just have to use:

      value = service[data_name]

Python script

A Python script may have methods declared for:

• The initialization of the module
• The start of the module
• The cyclic call for a cyclic service
• The subscription of a service

Module initialization

The Python script may have a method called at the initialization of the module with the following signature:

      def <method_name>(self, pythonUtils):

This method will be called when the module is initialized, during the framework initialization.

To declare the name of the method, you must put the associated entry point, for example:

      <pythonImplementation path="pythonAppli">
         <initEntryPoint method="myInitMethod" />
      </pythonImplementation>

Module start

The Python script may have a method called at the start of the module with the following signature:

      def <method_name>(self, pythonUtils):

This method will be called when the module is started, during the framework effective start.

To declare the name of the method, you must put the associated entry point, for example:

      <pythonImplementation path="pythonAppli">
         <startEntryPoint method="myStartMethod" />
      </pythonImplementation>

Module cyclic call

The Python script may have a method called by when invoking automatically a Service. The method should have with the following signature:

      def <method_name>(self, pythonUtils, serviceName):

This method will be called for a publish Services and event Services called cyclically.

To declare the name of the method, you must put the associated entry point, for example:

      <pythonImplementation path="pythonAppli">
         <defaultSendEntryPoint method="myCyclicMethod" />
      </pythonImplementation>

Service subscription

The Python module may have a subscribe method with the following signature^[9]

the subscribe name is the default name for all service subscriptions. But it is possible to change the method name for all Services or just for one Service. See

:

      def subscribe(self, pythonUtils, serviceName):

This method will be called when the module is notified from the invocation of any Service. To detect which service is at the origin of the notification, you can perform:

      # to get the service itself
      service = pythonUtils.getService(serviceName)

The datas of the service can be retrieved with:

      data = service[theDataName]

To set a value for a data for any Service for which the module is a provider, you can perform:

      pythonUtils.setValue(service, theDataName, theDataValue)

To invoke a Service for which the module is a provider, you can perform:

      pythonUtils.send(serviceName)

Example

In the following example, the FlightManagementSystem module:

• subscribe to the position publish service
• subscribe to the directTo event service
• provides the computeFlightPlan request-response service

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli"/>
         <interfaces>
            <subscribe service="position" />
            <eventReceived service="directTo"/>
            <requestReceived service="computeFlightPlan"/>
         </interfaces>
      </pythonModule>

Another way to specify the Python module is to explictly specify the ports and port sizes for the Java / Python communication:

      <pythonModule name="FlightManagementSystem" >
         <pythonImplementation path="pythonAppli" inputPort="6000" outputPort="6005" inputSize="1024" outputSize="1024"/>
         <interfaces>
            <subscribe service="position" />
            <eventReceived service="directTo"/>
            <requestReceived service="computeFlightPlan"/>
         </interfaces>
      </pythonModule>

We could have the following code for the script:

      from pythonUtils import PythonUtils

      class PythonAppli:
        def subscribe(self, pythonUtils, serviceName):
        service = pythonUtils.getService(serviceName)
        if serviceName == "position":
        // get position
        ...
        elif serviceName == "directTo":
        // compute directTo
        ...
        elif serviceName == "computeFlightPlan":
        // compute FlightPlan and sends the response
        ...