Log in
Site MapContact

18 Background Processing

In high volume labs, such as those testing for tens of pesticides per sample, system performance slows down for big tasks such as submitting big Worksheets for Verification, and it is better to perform these tasks asynchronously in the background, freeing the user up to continue with other tasks in the LIMS. Asynchronous processes complete quicker too

Configurable Switching

Small tasks containing little data are manageable in the UI and do not require any improvement by way of background processing and with bika.qmanager, the user is able to do either foreground or background processing depending on how much data needs to be processed in a task, by setting limits for the LIMS to switch to background processing

The limits are specified for the number of Analyses on a group of Samples or on Worksheets to be processed. The user executes the transitions, e.g. clicking the [Receive] button for a list of Samples, without having to consider which route to follow

For tasks smaller than the configured limits, normal synchronous processing proceeds without interrupting the users’ workflow and he/she is returned to the next page after the transition

When the number of Samples and Analysis submitted are more than the limits, the LIMS sends them to a queue for processing them asynchronously in the background while the user is returned to the LIMS with a ‘Transactions successfully queued’ message

Application

Queuing applies to Sample

    • Registration
    • Receipt
    • Cancellation

and for Worksheet Analyses

    • Assignment - both manually and using Worksheet Templates
    • Saving
    • Submission for Verification
    • Verification

Server Installation

For the server installation, please see the senaite.queue tech manual. When complete you should have two Zope clients, queue_server and queue_consumer listening on their individual ports for queuing requests

You also have to install bika.qmanager to set the limits for synchronous processing to divert to asynchronous in the background

Add a bash script to run queue_consume and use the port number of the Zope client set up for it

Create bash script and save it as cron_queue_site_name.sh on the path that will be used on the server's crontab: 

#!/bin/sh

for i in 1 2 3 4 5 6 7 8 9 10

do 

    /usr/bin/curl -u admin:adminpw 0.0.0.0:8683/yourSiteName/queue_consume

    sleep 5

done

Where:

admin                 is the admin user

adminpw            admin password

8683                  port of the queue_consume Zope client

yourSiteName   the name of the site

Edit the crontab, crontab -e and add for the job to run every minute:

MAILTO=’’

* * * * * /home/path/to/cron_queue_site_name.sh

Configuring Queue Management

In the Queue's Settings, point to the Queue server client and set the numbers of objects to process, max retries, the minimum seconds reserved per task and maximum seconds to wait

Set the Analysis caps for synchronous processing to diverge to the background and free the user up to continue with other tasks

Using Asynchronous Processing

Submitting tasks to the queue

Monitoring the queue