Skip to main content
Solved

Get all records from a table


vitoria36

Hi!

I'm trying to get all records from a table, but using table_records isn't giving me all of them.

Running the following code in Python I get 50 records, but my table has 94. Any ideas of how I can get all of them?

import requests
import json

payload = {"query": "{  table_records(table_id:\"TABLE_ID\") {edges {node {id }}}}"}

headers = {
    "authorization": f"Bearer {pipefy_token}",
    "content-type": "application/json"
}

response = requests.request("POST", url, json=payload, headers=headers)

json_data = json.loads(response.text)

print(len(json_data["data"]["table_records"]["edges"]))

 

Best answer by vitoria36

I managed to solve using pagination. Basically, each query returns a maximum of 50 objects, so it is necessary to make several queries to get all the records. I managed to do this with the answers to this question.

 

If someone ever needs to query all the records from a table and store them in a DataFrame in Python, here is my code:

import requests
import json
import pandas as pd

table_id = TABLE_ID
pipefy_token = TOKEN
table_field = TABLE_FIELDS

url = "https://api.pipefy.com/graphql"

headers = {
    "authorization": f"Bearer {pipefy_token}",
    "content-type": "application/json"
}

records_df = pd.DataFrame(columns=table_fields)

has_next_page = True
first_query = True

while(has_next_page):

    if first_query:
        payload = {"query": "{  table_records(table_id:\""+table_id+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}
        first_query = False
    else:
        payload = {"query": "{  table_records(table_id:\""+table_id+"\",after:\""+end_cursor+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}

    response = requests.request("POST", url, json=payload, headers=headers)

    json_data = json.loads(response.text)
    end_cursor = json_data["data"]["table_records"]["pageInfo"]["endCursor"]
    has_next_page = json_data["data"]["table_records"]["pageInfo"]["hasNextPage"]
    total_records_pg = len(json_data["data"]["table_records"]["edges"])

    for i in range(total_records_pg):
        card_title = json_data["data"]["table_records"]["edges"][i]["node"]["title"]
        card_data_d = json_data["data"]["table_records"]["edges"][i]["node"]["record_fields"]
        card_data = {x['name']:x['value'] for x in card_data_d}
        records_df = records_df.append(card_data, ignore_index=True)

 

View original
Did this topic help you find an answer to your question?

2 replies

vitoria36
  • Author
  • Regular Participant
  • 2 replies
  • Answer
  • May 27, 2021

I managed to solve using pagination. Basically, each query returns a maximum of 50 objects, so it is necessary to make several queries to get all the records. I managed to do this with the answers to this question.

 

If someone ever needs to query all the records from a table and store them in a DataFrame in Python, here is my code:

import requests
import json
import pandas as pd

table_id = TABLE_ID
pipefy_token = TOKEN
table_field = TABLE_FIELDS

url = "https://api.pipefy.com/graphql"

headers = {
    "authorization": f"Bearer {pipefy_token}",
    "content-type": "application/json"
}

records_df = pd.DataFrame(columns=table_fields)

has_next_page = True
first_query = True

while(has_next_page):

    if first_query:
        payload = {"query": "{  table_records(table_id:\""+table_id+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}
        first_query = False
    else:
        payload = {"query": "{  table_records(table_id:\""+table_id+"\",after:\""+end_cursor+"\") {edges {node {id title record_fields {name value}}} pageInfo {endCursor hasNextPage}}}"}

    response = requests.request("POST", url, json=payload, headers=headers)

    json_data = json.loads(response.text)
    end_cursor = json_data["data"]["table_records"]["pageInfo"]["endCursor"]
    has_next_page = json_data["data"]["table_records"]["pageInfo"]["hasNextPage"]
    total_records_pg = len(json_data["data"]["table_records"]["edges"])

    for i in range(total_records_pg):
        card_title = json_data["data"]["table_records"]["edges"][i]["node"]["title"]
        card_data_d = json_data["data"]["table_records"]["edges"][i]["node"]["record_fields"]
        card_data = {x['name']:x['value'] for x in card_data_d}
        records_df = records_df.append(card_data, ignore_index=True)

 


william-kliemann
what information do I fill in the "TABLE-FIELDS" field?
 

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings