Datatable Import
The datatable import service is to allow the import of large amounts of data into existing datatables.
fetchRecent
array datatable_import.fetchRecent( int datatableId )Fetch information about recent datatable imports.
Parameters
datatableId
ID of the datatable.
Return Values
Returns an indexed array of associative arrays for each import. The index is returned in reverse chronological order.
Key | Type | Comment |
|---|---|---|
import_id | string | unique identifier for the import |
status | string | status of the import |
progress | int | import percentage complete |
total_rows | string | total rows in the CSV |
processed_rows | string | number of records that have been processed |
successful_rows | string | number of new records that were successfully imported |
duplicate_rows | int | number of records that already existed |
failed_rows | int | number of rows that failed to import |
started_ts | timestamp | datetime that the import was started |
finished_ts | timestamp | datetime that the import completed |
Note the difference between rows and records in the row count values.
A row is a line in the text of a CSV file.
A record is a single line of CSV data.
processed_rows = successful_rows + duplicate_rows + failed_rows
However, if a record contains a field with line-breaks, then even when progress == 100 :
total_rows != processed_rows
importUploadedFile
int datatable_import.importUploadedFile( int datatableId, string uploadKey, array importMapping, array importOptions )This method enables you to import a file based on an uploaded file key.
Parameters
datatableId
ID of the datatable to which you're importing.
uploadKey
The upload key of the file to import, see File_Upload.
importMapping
Associative array of column to profile field mapping.
If passing an empty array, the system will attempt to auto-detect the import mapping based on datatable field names present in the CSV header row. Headers must be enabled in the CSV options for this to work. All fields must be mapped, unless setting the autodetect.ignore_missing option to true.
Type | Description | |
|---|---|---|
Key | int | CSV column ID |
Value | int | Datatable field ID |
importOptions
An associative array specifying additional options for the import process. Various options are available:
Key | Required | Type | Description |
|---|---|---|---|
csv | Yes | array | See below |
prune | No | boolean | Default false. |
notify | No | array | See below |
autoDetect | No | array | See below |
csv
An associative array of CSV options:
Key | Required | Type | Description |
|---|---|---|---|
has_headers | Yes | boolean | Specifies whether the first row describes the columns |
delimiter | No | char | The CSV data's field separator character. Defaults to comma |
enclosure | No | char | The CSV data's text enclosing character. Defaults to double quote |
notify
An associative array of notification options. Does not need to be set, but if defined must include both keys described below.
Key | Type | Description |
|---|---|---|
type | string | 'all' or 'error'. Defines the types of notification to send. |
string | Comma-separated list of email addresses |
autoDetect
An associative array of auto-detect options. All keys are optional.
Key | Type | Description |
|---|---|---|
ignore_missing | boolean | true to ignore CSV columns which are not mapped to a valid datatable field. Default false. |
Return Values
Returns unique identifier for the import process.
getProgress
string datatable_import.getProgress( int importId )Returns a range of information about any current or complete import progress. Information includes percent complete, standard statistics, timestamps of the start and end of the import and the estimated time of arrival.
Parameters
importId
ID of the import of which you want to retrieve the progress.
Return Values
Returns an array of available and calculated information. See below for an example.
array(
'percent' => float,
'statistics' => array(
'total_rows' => integer,
'processed_rows' => integer,
'successful_rows' => integer,
'failed_rows' => integer,
'duplicate_rows' => integer
),
'timestamps' => array(
'unix' => array(
'started' => integer,
'finished' => integer
),
'database' => array(
'started' => string,
'finished' => string
)
),
'eta' => array(
'seconds' => integer,
'breakdown' => array(
'days' => integer,
'hours' => integer,
'minutes' => integer,
'seconds' => integer
)
)
)Service
datatable_import
Methods