Skip to content

HTTP Write

The Write API endpoint is /db/write/{TABLE}, where {TABLE} is the name of the table you want to write to.

Although the query API can also execute an INSERT statement, it is not an efficient way to write data, since clients must build a static SQL text in the q parameter for every request. The proper way to write data is the write API, which is equivalent to the INSERT statement. Another benefit of write is that a client application can insert multiple records in a single write request.

Parameters

Write Parameters

paramdefaultdescription
timeformatnsTime unit: s, ms, us, ns
tzUTCTime zone: UTC, Local, or a location name
methodinsertWriting method: insert, append
dbMACHBASEDBTarget database name for multiple-database environments. Since v8.7.0

INSERT vs. APPEND

By default, the /db/write API uses the INSERT INTO... statement to write data. For a small number of records, this method performs similarly to the append method.

When writing a large amount of data (e.g., more than several hundred thousand records), use the method=append parameter. This tells Machbase Neo to use the append method instead of the default INSERT INTO... statement, which is implicitly specified as method=insert.

Multiple-Database

When the server hosts more than one named database, use the db query parameter to select the target database for the write, for both method=insert and method=append. For a JSON body request, the same target can be given as a top-level "db" field instead of (or together with) the query parameter; the query parameter takes precedence when both are given.

If db is omitted or empty, the write targets the default database MACHBASEDB. An invalid database name responds with 400 Bad Request, and a database that does not exist (or that the connected user cannot access) responds with an error.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?db=OTHERDB
Content-Type: application/json

{
    "data": {
        "columns":["name", "time", "value"],
        "rows": [
            [ "json-data", 1670380342000000000, 1.0001 ]
        ]
    }
}
```

Content-Type Header

The machbase-neo server recognizes the format of the incoming data stream by the Content-Type header. For example, use Content-Type: application/json for JSON data, Content-Type: text/csv for CSV data, and Content-Type: application/x-ndjson for newline-delimited JSON data.

Content-Encoding Header

If a client sends a gzip-compressed stream, it should set the Content-Encoding: gzip header to tell machbase-neo that the incoming data stream is encoded in gzip.

Inputs

JSON

This request message has the same structure as the SQL statement INSERT INTO {table} (columns...) VALUES (values...).

nametypedescription
dataobjectData body
data.columnsarray of stringsList of columns
data.rowsarray of tuplesArray of record values

JSON

{
    "data": {
        "columns":["name", "time", "value"],
        "rows": [
            [ "json-data", 1670380342000000000, 1.0001 ],
            [ "json-data", 1670380343000000000, 2.0002 ]
        ]
    }
}

Set the Content-Type header to application/json.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
Content-Type: application/json

{
    "data": {
        "columns":["name", "time", "value"],
        "rows": [
            [ "json-data", 1670380342000000000, 1.0001 ],
            [ "json-data", 1670380343000000000, 2.0002 ]
        ]
    }
}
```

Compressed JSON

If the incoming stream is gzip-compressed, set the Content-Encoding: gzip header to tell machbase-neo.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
Content-Type: application/json
Content-Encoding: gzip

< /csv/post-data.json.gz
```

JSON with timeformat

If the time fields are strings instead of UNIX epoch values, add the timeformat and tz parameters.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=DEFAULT
    &tz=Asia/Seoul
Content-Type: application/json

{
    "data": {
        "columns":["name", "time", "value"],
        "rows": [
            [ "json-data", "2022-12-07 02:32:22", 1.0001 ],
            [ "json-data", "2022-12-07 02:32:23", 2.0002 ]
        ]
    }
}
```

NDJSON

NDJSON (Newline Delimited JSON) is a format for streaming JSON data where each line is a valid JSON object. This is useful for processing large datasets or streaming data.

This request message has the same structure as the SQL statement INSERT INTO {table} (columns...) VALUES (values...).

NDJSON

{"NAME":"ndjson-data", "TIME":1670380342000000000, "VALUE":1.001}
{"NAME":"ndjson-data", "TIME":1670380343000000000, "VALUE":2.002}

Set the Content-Type header to application/x-ndjson.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
Content-Type: application/x-ndjson

{"NAME":"ndjson-data", "TIME":1670380342000000000, "VALUE":1.001}
{"NAME":"ndjson-data", "TIME":1670380343000000000, "VALUE":2.002}
```

NDJSON with timeformat

If the time fields are strings instead of UNIX epoch values, write them as follows.

{"NAME":"ndjson-data", "TIME":"2022-12-07 02:33:22", "VALUE":1.001}
{"NAME":"ndjson-data", "TIME":"2022-12-07 02:33:23", "VALUE":2.002}

Add the timeformat and tz parameters.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=DEFAULT
    &tz=Local
Content-Type: application/x-ndjson

{"NAME":"ndjson-data", "TIME":"2022-12-07 02:33:22", "VALUE":1.001}
{"NAME":"ndjson-data", "TIME":"2022-12-07 02:33:23", "VALUE":2.002}
```

CSV

These options are only applicable when the content body is in CSV format.

paramdefaultdescription
headerskip: skip the first line
columns: the header line fields match the table column names.
delimiter,Field delimiter

If the CSV data includes a header line, set the header=skip query parameter so that machbase-neo ignores the first line.

If the CSV header line specifies the columns to write, use header=columns. The header must match the column names of the table, and the header line is used as the columns part of the SQL statement INSERT INTO TABLE(columns...) VALUES(...).

If the header line is not included and the header option is omitted, the fields of each line must match all the columns of the table in order, because the data is written as the SQL statement INSERT INTO TABLE VALUES(...).

header=skip

If you set header=skip, the server ignores the first line, and the data must be in the same order as the columns of the table.

NAME,TIME,VALUE
csv-data,1670380342000000000,1.0001
csv-data,1670380343000000000,2.0002

The Content-Type header should be text/csv.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?header=skip
Content-Type: text/csv

NAME,TIME,VALUE
csv-data,1670380342000000000,1.0001
csv-data,1670380343000000000,2.0002
```

header=columns

If the CSV fields are in a different order or are a subset of the actual table columns, set header=columns. The server then treats the first line as the column names. The example below internally generates an SQL statement equivalent to INSERT INTO EXAMPLE (TIME, NAME, VALUE) VALUES(?, ?, ?).

TIME,NAME,VALUE
1670380342000000000,csv-data,1.0001
1670380343000000000,csv-data,2.0002

The Content-Type header should be text/csv.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?header=columns
Content-Type: text/csv

TIME,NAME,VALUE
1670380342000000000,csv-data,1.0001
1670380343000000000,csv-data,2.0002
```

Compressed CSV

If the incoming stream is gzip-compressed, set the Content-Encoding: gzip header to tell machbase-neo.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?header=skip
Content-Type: text/csv
Content-Encoding: gzip

< /csv/post-data.csv.gz
```

CSV with timeformat

Add the timeformat and tz query parameters.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?header=skip
    &timeformat=Default
    &tz=Asia/Seoul
Content-Type: text/csv

NAME,TIME,VALUE
csv-data,2022-12-07 11:39:32,1.0001
csv-data,2022-12-07 11:39:33,2.0002
```

Examples

For details of the API, refer to Request endpoint and parameters.

Test Table

```http
GET http://127.0.0.1:5654/db/query
    ?q=create tag table if not exists EXAMPLE (name varchar(40) primary key, time datetime basetime, value double)
```

Time

The time values in the sample files used in these examples are UNIX epoch values in seconds. Therefore, specify the timeformat=s option when loading the data. If the data is stored in a different resolution, change this option to match. Machbase Neo assumes nanoseconds (ns) as the default time resolution.

JSON with epoch

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=s
Content-Type: application/json

{
  "data":  {
    "columns":["NAME","TIME","VALUE"],
    "rows": [
        ["wave.sin",1676432361,0],
        ["wave.sin",1676432362,0.406736],
        ["wave.sin",1676432363,0.743144],
        ["wave.sin",1676432364,0.951056],
        ["wave.sin",1676432365,0.994522]
    ]
  }
}
```

Select rows

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
```

CSV with epoch

If the CSV data has a header line like below, set the header=skip query parameter.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=s
    &header=skip
Content-Type: text/csv

NAME,TIME,VALUE
wave.sin,1676432361,0.000000
wave.cos,1676432361,1.000000
wave.sin,1676432362,0.406736
wave.cos,1676432362,0.913546
wave.sin,1676432363,0.743144

```

CSV without header

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?timeformat=s
Content-Type: text/csv

wave.sin,1676432361,0.000000
wave.cos,1676432361,1.000000
wave.sin,1676432362,0.406736
wave.cos,1676432362,0.913546
wave.sin,1676432363,0.743144
```

CSV

Insert

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=Default
Content-Type: text/csv

wave.sin,2023-02-15 03:39:21,0.111111
wave.sin,2023-02-15 03:39:22.111,0.222222
wave.sin,2023-02-15 03:39:23.222,0.333333
wave.sin,2023-02-15 03:39:24.333,0.444444
wave.sin,2023-02-15 03:39:25.444,0.555555
```
```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
    &timeformat=Default
    &format=csv
```

Append

When loading a large CSV file, the append method can write data several times faster than the insert method.

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE?timeformat=s&method=append
Content-Type: text/csv

wave.sin,1676432361,0.000000
wave.cos,1676432361,1.000000
wave.sin,1676432362,0.406736
wave.cos,1676432362,0.913546
wave.sin,1676432363,0.743144
```

CSV with time zone

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=Default
    &tz=Asia/Seoul
Content-Type: text/csv

wave.sin,2023-02-15 12:39:21,0.111111
wave.sin,2023-02-15 12:39:22.111,0.222222
wave.sin,2023-02-15 12:39:23.222,0.333333
wave.sin,2023-02-15 12:39:24.333,0.444444
wave.sin,2023-02-15 12:39:25.444,0.555555
```

Select rows in UTC

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
    &timeformat=Default
    &format=csv
```

RFC3339

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=RFC3339
Content-Type: text/csv

wave.sin,2023-02-15T03:39:21Z,0.111111
wave.sin,2023-02-15T03:39:22Z,0.222222
wave.sin,2023-02-15T03:39:23Z,0.333333
wave.sin,2023-02-15T03:39:24Z,0.444444
wave.sin,2023-02-15T03:39:25Z,0.555555
```

Select rows in UTC

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
    &format=csv
    &timeformat=RFC3339
```

RFC3339Nano in time zone

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=RFC3339Nano
    &tz=America/New_York
Content-Type: text/csv

wave.sin,2023-02-14T22:39:21.000000000-05:00,0.111111
wave.sin,2023-02-14T22:39:22.111111111-05:00,0.222222
wave.sin,2023-02-14T22:39:23.222222222-05:00,0.333333
wave.sin,2023-02-14T22:39:24.333333333-05:00,0.444444
wave.sin,2023-02-14T22:39:25.444444444-05:00,0.555555
```

Select rows in America/New_York

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
    &format=box
    &timeformat=RFC3339Nano
    &tz=America/New_York
```

Timeformat

```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=Default
Content-Type: text/csv

wave.sin,2023-02-15 03:39:21,0.111111
wave.sin,2023-02-15 03:39:22.111111111,0.222222
wave.sin,2023-02-15 03:39:23.222222222,0.333333
wave.sin,2023-02-15 03:39:24.333333333,0.444444
wave.sin,2023-02-15 03:39:25.444444444,0.555555
```

Select rows in UTC

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 10
    &format=csv
    &timeformat=Default

```

Custom Timeformat

  • hour:min:sec-SPLIT-year-month-day format in the New York time zone
```http
POST http://127.0.0.1:5654/db/write/EXAMPLE
    ?timeformat=03:04:05.999999999-SPLIT-2006-01-02
    &tz=America/New_York
Content-Type: text/csv

wave.sin,10:39:21-SPLIT-2023-02-14 ,0.111111
wave.sin,10:39:22.111111111-SPLIT-2023-02-14 ,0.222222
wave.sin,10:39:23.222222222-SPLIT-2023-02-14 ,0.333333
wave.sin,10:39:24.333333333-SPLIT-2023-02-14 ,0.444444
wave.sin,10:39:25.444444444-SPLIT-2023-02-14 ,0.555555
```

Select rows

```http
GET http://127.0.0.1:5654/db/query
    ?q=select * from EXAMPLE limit 5
    &format=csv
    &timeformat=2006-01-02 03:04:05.999999999
    &tz=America/New_York
```
Last updated on