Skip to content

Commit 644bd42

Browse files
eliblocksimonw
andauthored
feat: set user agent with --user-agent flag (#12)
* feat: set user agent with --user-agent flag * docs: move request options after export options Co-authored-by: Simon Willison <swillison@gmail.com>
1 parent e7b0e13 commit 644bd42

2 files changed

Lines changed: 19 additions & 6 deletions

File tree

README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,14 @@ The SQLite database will have a table created for each table you export. Those t
5555

5656
If you run this command against an existing SQLite database records with matching primary keys will be over-written by new records from the export.
5757

58+
## Request options
59+
60+
By default the tool uses [python-httpx](https://www.python-httpx.org)'s default configurations.
61+
62+
You can override the `user-agent` using the `--user-agent` option:
63+
64+
airtable-export export base_id table1 table2 --key=key --user-agent "Airtable Export Robot"
65+
5866
## Running this using GitHub Actions
5967

6068
[GitHub Actions](https://github.com/features/actions) is GitHub's workflow automation product. You can use it to run `airtable-export` in order to back up your Airtable data to a GitHub repository. Doing this gives you a visible commit history of changes you make to your Airtable data - like [this one](https://github.com/natbat/rockybeaches/commits/main/airtable).

airtable_export/cli.py

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
)
2424
@click.argument("tables", type=str, nargs=-1)
2525
@click.option("--key", envvar="AIRTABLE_KEY", help="Airtable API key", required=True)
26+
@click.option("--user-agent", help="User agent to use for requests")
2627
@click.option("-v", "--verbose", is_flag=True, help="Verbose output")
2728
@click.option("--json", is_flag=True, help="JSON format")
2829
@click.option("--ndjson", is_flag=True, help="Newline delimited JSON format")
@@ -32,7 +33,9 @@
3233
type=click.Path(file_okay=True, dir_okay=False, allow_dash=False),
3334
help="Export to this SQLite database",
3435
)
35-
def cli(output_path, base_id, tables, key, verbose, json, ndjson, yaml, sqlite):
36+
def cli(
37+
output_path, base_id, tables, key, user_agent, verbose, json, ndjson, yaml, sqlite
38+
):
3639
"Export Airtable data to YAML file on disk"
3740
output = pathlib.Path(output_path)
3841
output.mkdir(parents=True, exist_ok=True)
@@ -48,7 +51,7 @@ def cli(output_path, base_id, tables, key, verbose, json, ndjson, yaml, sqlite):
4851
records = []
4952
try:
5053
db_batch = []
51-
for record in all_records(base_id, table, key):
54+
for record in all_records(base_id, table, key, user_agent=user_agent):
5255
r = {
5356
**{"airtable_id": record["id"]},
5457
**record["fields"],
@@ -89,17 +92,19 @@ def cli(output_path, base_id, tables, key, verbose, json, ndjson, yaml, sqlite):
8992
)
9093

9194

92-
def all_records(base_id, table, api_key, sleep=0.2):
95+
def all_records(base_id, table, api_key, sleep=0.2, user_agent=None):
96+
headers = {"Authorization": "Bearer {}".format(api_key)}
97+
if user_agent is not None:
98+
headers["user-agent"] = user_agent
99+
93100
first = True
94101
offset = None
95102
while first or offset:
96103
first = False
97104
url = "https://api.airtable.com/v0/{}/{}".format(base_id, quote(table))
98105
if offset:
99106
url += "?" + urlencode({"offset": offset})
100-
response = httpx.get(
101-
url, headers={"Authorization": "Bearer {}".format(api_key)}
102-
)
107+
response = httpx.get(url, headers=headers)
103108
response.raise_for_status()
104109
data = response.json()
105110
offset = data.get("offset")

0 commit comments

Comments
 (0)