Suppliers

Please read this document carefully. It contains important information on how to submit leads and the different methods that are available in order to receive key information about those leads.

Submitting Leads via HTTP POST


Leads are to be submitted by making a HTTP POST request to:

https://[instance name].databowl.com/api/v1/lead
Where [instance name] is the name of the instance provided to you.

Important: HTTPS must be used and the request must be POST otherwise the lead will not be accepted.
The request must be a valid HTTP POST request. Therefore you must make sure that the request contains appropriate HTTP headers. As a minimum you must include the following headers:
  • Content-Length
  • Content-Type (Which must equal application/x-www-form-urlencoded)
  • Host

Post Parameters

All data is to be submitted in the request body in the application/x-www-form-urlencoded format. The following parameters are always required:

Parameter Type Description
cid integer The id of the campaign. This must be the actual id, not reference or name.
sid integer Your supplier id. This must be the actual id, not reference or name.

The actual lead data is to be submitted using the following naming convention:

f_<field definition id>_<lowercase field label>

All required fields for the campaign must be included in the request otherwise the lead will be rejected. Fields that are not required may be completely omitted from the request or left blank. Any fields submitted that are not part of the campaign will be silently ignored.

Important: All data must use UTF-8 encoding. Leads may be rejected or ignored if UTF-8 is not used.

Important: The POST data must be placed in the request body. Placing the data in the query string is not acceptable

Response

The API will return a response in the JSON format with the following structure:

{
	"result": "<result>",
	"lead_id":"<lead_id>",
	"error": {
		"msg": "<error message>",
		...<additional_fields>...
	}
}
	

Result will either be "created" or "error". The "error" key in the returned JSON will only be present if the result is "error". You should store the lead_id in order to reference the lead.

Important: A result of "created" does not indicate that the lead has been accepted or rejected only that it has passed an initial set of checks and has been submitted for further processing.

Errors

If the lead has a result of "error" then error message will be one of the following:

Message Description
MISSING_CID

The cid (campaign_id) was missing from the request.

Note, if you are receiving this message but think you are already including the cid then please make sure you are submitting data as set out in this documentation.

INVALID_CID The cid (campaign_id) supplied in the request was invalid.
MISSING_SID The sid (supplier_id) was missing from the request.
INVALID_SID The sid (supplier_id) supplied in the request was invalid.
SUPPLIER_NOT_VALID_FOR_CAMPAIGN You are not able to submit leads for the campaign you specified in the request because you are not an assigned supplier for that campaign.
MISSING_FIELD A required field is missing from the request.
DUPLICATE_LEAD The lead you are submitted has been identified as a duplicate and, therefore, has been rejected.
TOTAL_CAP_REACHED Either the total cap on the campaign or your supplier-specific total cap for the campaign has been reached. Therefore no further leads will be accepted until this limit is manually raised.
DAILY_CAP_REACHED Either the daily cap on the campaign or your supplier-specific daily cap for the campaign has been reached for the day. It will not be possible to submit further leads until tomorrow or if the cap is manually raised.
TOTAL_CAP_REACHED_QUARANTINED Either the total cap on the campaign or your supplier-specific total cap for the campaign has been reached. The lead has not currently been accepted but may be accepted at a later date.
DAILY_CAP_REACHED_QUARANTINED Either the daily cap on the campaign or your supplier-specific daily cap for the campaign has been reached for the day. The lead has not currently been accepted but may be accepted at a later date.
SUPPLIER_PAUSED You are currently paused on the campaign you are attempting to supply the lead for. The lead has not currently been accepted but may be accepted at a later date.

Submitting leads via FTP


It is also possible to submit leads via FTP in CSV format. The FTP server is running on port 21 on ftp.databowl.com. In order for the submission of leads to be successful there are a number of important conventions that must be followed.

CSV Format

Every CSV file uploaded must obey the following rules:

  • Fields must be separated by a single comma delimiter.
  • Fields may be enclosed in a pair of double quotation marks ("), if required (e.g if the field contains a comma).
  • If it is required to escape a character (e.g if a quoted field contains a double quotation mark) then the character to be escaped must be prefixed with a single backslash.
  • All rows must have the same number of fields.
  • The first line of the file must contain a valid header (more details below)
Danger: If a file that is in anyway invalid is uploaded the whole the file, or any part thereof, may be rejected without notice.

Every CSV file uploaded must start with a valid header. The header describes what each field in the CSV file contains and must be named according to the field labels used in the campaign.

For example, if the campaign had the following fields with labels: firstname, lastname, email, dob then the CSV file header would be:

firstname,lastname,email,dob
	

With the following rows corresponding to the field order defined in the header. If a field has an invalid name or it otherwise misnamed it will be ignored. If this results in a required field being missing then all the leads may be rejected.

Uploading to FTP Server

The FTP server requires use of SSL/TLS in order to protect your login credentials and any files you upload. Your FTP client may refer to this as explicit FTP over TLS/SSL or FTP with TLS/SSL, you should not select implicit SSL/TLS otherwise the connection will fail.

If you receive the error message: 530 Non-anonymous sessions must use encryption then you have not configured your FTP client to use TLS/SSL.

Credentials

You must supply valid credentials in order to login to the FTP server. By default you should login using the same credentials as the Databowl web interface with the username in the following format:

<instance_name>-<username>

For example, if your username was John and the instance was called company then your username would be company-John with the password being as normal.

File naming and Location

All files must be placed in the lead_uploads directory on the FTP server. You will not be able to upload files anywhere else. Additionally, while you can upload files to this directory you cannot see the contents of the directory, this is intended. If your FTP client reports: 226 Transfer done (but failed to open directory) you can safetly ignore this.

Files must follow the following naming convention:

<instance_name>_<supplier_id>_<campaign_id>_<anything>.csv
Danger: Files that do not comply with the naming convention or include invalid values will be rejected with no notification or warning.
Warning: You will not be able to see the file you have uploaded in the directory listing, this is intended. If the file transfer completed successfully, the file was named correctly and was in the correct format it will be processed. Do not repeatedly upload the same file.

Receiving Lead Updates


Once a lead has been submitted via the API the status of the lead will be updated a number of times as it is processed. You are able to receive notifications of these status updates by a number of methods.

Statuses

The lead status will be returned as an integer. The following table maps the status identifiers to human readable status names:

Status ID Name Description
1 Received The lead has been conditionally accepted pending response from client.
2 Rejected The lead has been rejected.
3 Flagged There are one or more potential problems with the lead. This is only a temporary status while the lead is either automatically or manually reviewed.
4 Pending The lead has been received and is queued for further checks. This status is only temporary.
5 Quarantined The lead is not currently accepted due a cap being reached. However, the lead may be accepted at a later date. Not applicable to all campaigns
6 Accepted The lead has been accepted. Note, this may be conditionally.
7 Client Rejected The lead has been rejected by the client.

Receiving Lead Updates using AMQP (Recommended)

You can receive lead updates in real time by subscribing to your supplier specific queue in our AMQP broker. Whenever something about a lead changes the lead will be routed to your supplier specific queue allowing you to receive real time updates.

The AMQP broker is running on port 5672 and is accessible via [instance name].databowl.com. Please note that the broker is behind a load balancer that has a TCP idle timeout of 60 seconds. You should use AMQP's heartbeat functionality to prevent persistent TCP connections from timing out.

Authentication is required to connect to the broker, if you do not have your credentials please contact us.

The name of the queue that will be created for you will be in the following format:

lead_feed_sid_<your supplier id>

Your account is restricted to consuming messages from this queue only. You should not attempt to publish any messages, consume messages from another queue or (re)declare any exchanges or queues.

To prevent unnecessary server load the queue will automatically be deleted after a period of 2 hours of inactivity. You should make a HTTP POST request to:

https://[instance name].databowl.com/api/v1/lead_feed/request_queue

Passing a sid parameter, with your supplier id. to create the queue initially. If you do not consume from the queue in 2 hours then you must make the HTTP POST request before attempting to consume from the queue again.

In addition to the queue timeout there is a message TTL of one hour on every message published to the queue. This means if you do not consume the lead notification within an hour of it occurring it will be removed from the queue.

A message will be routed to your supplier queue every time a lead is processed in our system which was supplied by you. There is no fixed amount of times a lead may be processed and can be reprocessed manually at any time. The message will contain the latest version of the data and metadata we hold about the lead with subsequent messages taking precedent over any previous messages.

How you consume from the queue is up to you. However, you should keep in mind the restrictions in how long the messages in the queue and the queue itself will persist if the queue is left inactive. If you use basic.consume and keep a constant channel open with the broker then you should never miss any messages. Although if you decide to use basic.get then you should make sure you poll the queue at a frequent enough interval to prevent the loss of messages.

Important: Please ensure you acknowledge the messages you receive using basic.ack otherwise the broker will attempt to redeliver the unacknowledged messages until they expire.

Polling for Lead Updates using HTTP GET

If you are unable to use AMQP directly then you can issue a HTTP GET request to an API that will fetch messages from the AMQP queue for you and put them in a JSON array. The same restrictions apply to using the HTTP GET API as they would if you were connecting directly to the AMQP broker. Therefore if you have not consumed any messages via the API or directly from the broker within the last 2 hours then you must issue a HTTP POST request to:

https://[instance name].databowl.com/api/v1/lead_feed/request_queue

Where [instance name] is the name of the instance provided to you.

Passing the following parameters:

Parameter Description
sid Your supplier id
key Your public API key.
timestamp Current timestamp (GMT)
signature Calculated signature, see authentication below.

In addition, messages will still expire after they have been in the queue for an hour. You should poll the API frequently enough to make sure you don't miss any messages.

To poll for messages issue a HTTP GET request to:

https://[instance name].databowl.com/api/v1/lead_feed/get_messages

Where [instance name] is the name of the instance provided to you.

With the following query string parameters:

Parameter Description
sid Your supplier id
limit Max amount of messages to return. You cannot request more than 100 at a time.
key Your public API key.
timestamp Current timestamp (GMT)
signature Calculated signature, see authentication below.

Either a JSON array will be returned with any messages, up to the limit, that were in the queue or an empty JSON array if no messages were waiting.

Important: The API will automatically acknowledge the messages in the queue as it returns them in the request. This means the message will only ever be returned once by the API and if consumed by the API will not be sent to any consumers connecting directly to the AMQP broker.

Authentication

In order to receive lead updates via the HTTP GET API you must authenticate yourself. In order to authenticate you will need both your public and private API key which can be provided on request, or, if you have access, generated in Databowl.

To authenticate you need to pass three query parameters to the API: key, timestamp, and signature. The key parameter is your public key, timestamp the current number of seconds since the unix epoch (GMT), and signature is a SHA-256 HMAC hash generated as described below.

To generate the value for signature you should first take the sid, limit (not used in request_queue API), and timestamp parameters and format them as below:

limit=<limit>&sid=<sid>&timestamp=<timestamp>

Then compute the HMAC of this string using SHA-256 as the hashing algorithm and your private API key as the key. The result of this must be repesented in lowercase hexits. This result then can be used as the signature parameter.

For example with a sid of 45, a limit of 100 and a timestamp of 1391437190 the string would be:

limit=100&sid=45&timestamp=1391437190

(n.b if you omit limit then you should also omit the & before sid)

Then computing the HMAC using SHA-256 as the hashing algorithm and my private API key of: bf8edb8b94f442340b5e812717728464 would result in the following signature:

c93a467f4e57210334e6c77ef4cf41b695b470933fded162ad10360a61de170a

Message Format

When you receive messages from the queue they will be in JSON format with the following structure:

{
	"lead_id":"<lead_id>",
	"camp_id":"<campaign_id>",
	"supplier_id":"<supplier_id>",
	"status": "<status_id>",
	"data": {
		...<field_data>..
.	}
}
	
Parameter Description
lead_id The unique identifier for the lead. This will correspond to the lead_id returned when the lead was submitted over the API.
camp_id The identifier for the campaign that the lead was submitted to.
supplier_id The identifier for the supplier who submitted the lead.
status_id An integer that indicates the status of the lead. See the Statuses table for more information.
field_data Contains key value pairs that hold the actual lead data. The keys correspond to the field definition id.