Solved

Get all records from a table

  • 27 May 2021
  • 2 replies
  • 975 views

Userlevel 2

Hi!

I'm trying to get all records from a table, but using table_records isn't giving me all of them.

Running the following code in Python I get 50 records, but my table has 94. Any ideas of how I can get all of them?

import requests
import json

payload = {"query": "{ table_records(table_id:\"TABLE_ID\") {edges {node {id }}}}"}

headers = {
"authorization": f"Bearer {pipefy_token}",
"content-type": "application/json"
}

response = requests.request("POST", url, json=payload, headers=headers)

json_data = json.loads(response.text)

print(len(json_data["data"]["table_records"]["edges"]))

 

icon

Best answer by vitoria36 27 May 2021, 16:50

View original

2 replies

Userlevel 2

I managed to solve using pagination. Basically, each query returns a maximum of 50 objects, so it is necessary to make several queries to get all the records. I managed to do this with the answers to this question.

 

If someone ever needs to query all the records from a table and store them in a DataFrame in Python, here is my code:

import requests
import json
import pandas as pd

table_id = TABLE_ID
pipefy_token = TOKEN
table_field = TABLE_FIELDS

url = "https://api.pipefy.com/graphql"

headers = {
"authorization": f"Bearer {pipefy_token}",
"content-type": "application/json"
}

records_df = pd.DataFrame(columns=table_fields)

has_next_page = True
first_query = True

while(has_next_page):

if first_query:
payload = {"query": "{ table_records(table_id:\""+table_id+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}
first_query = False
else:
payload = {"query": "{ table_records(table_id:\""+table_id+"\",after:\""+end_cursor+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}

response = requests.request("POST", url, json=payload, headers=headers)

json_data = json.loads(response.text)
end_cursor = json_data["data"]["table_records"]["pageInfo"]["endCursor"]
has_next_page = json_data["data"]["table_records"]["pageInfo"]["hasNextPage"]
total_records_pg = len(json_data["data"]["table_records"]["edges"])

for i in range(total_records_pg):
card_title = json_data["data"]["table_records"]["edges"][i]["node"]["title"]
card_data_d = json_data["data"]["table_records"]["edges"][i]["node"]["record_fields"]
card_data = {x['name']:x['value'] for x in card_data_d}
records_df = records_df.append(card_data, ignore_index=True)

 

Userlevel 1
what information do I fill in the "TABLE-FIELDS" field?
 

 

 

Reply