Hi everyone,

I’m currently working on a Python script to automate report exports from Pipefy. I’m using the exportPipeReport and pipeReportExport GraphQL queries to export reports, retrieve download URLs, and save the files locally. However, I’ve encountered an issue where the download URLs sometimes work and sometimes don’t. Specifically, when they don’t work, I receive a 404 error. The odd part is that if I take the same URL and try it in a browser, it sometimes works but often fails.

Here’s the code I’m currently using:

import requests
import os
import time

# Data for the reports
reports = {
    "MASIVOS VH": {"pipe_id": 303931430, "report_id": 300754311, "name": "ventas_vh_reporte.xlsx"},
    "RENOVACIONES": {"pipe_id": 304179178, "report_id": 300757125, "name": "renovacion_es_reporte.xlsx"},
    "CANAL INDIRECTO": {"pipe_id": 304130823, "report_id": 300756746, "name": "canal_indirecto_reporte.xlsx"}
}

# Folder where files will be saved
SAVE_FOLDER = r"C:\Users\MateoEcheverría\OneDrive - ChevySeguro\Documentos\Somos Más\Bases de Carga (VH-CI-RENOV)"

# Function to export the report
def export_report(pipe_id, report_id):
    headers = {
        "Authorization": f"Bearer {TOKEN_PIPEFY}",
        "Content-Type": "application/json"
    }

    mutation = f"""
    mutation {{
      exportPipeReport(input: {{pipeId: {pipe_id}, pipeReportId: {report_id}}}) {{
        pipeReportExport {{
          id
        }}
      }}
    }}
    """

    response = requests.post("https://app.pipefy.com/graphql", headers=headers, json={'query': mutation})
    if response.status_code == 200:
        data = response.json()
        if data and "data" in data and datai"data"]a"exportPipeReport"]:
            return data "data"]u"exportPipeReport"]p"pipeReportExport"]p"id"]
    print("Error exporting the report.")
    return None

# Function to get the download URL with retry logic and exponential backoff
def get_download_url(export_id, attempts=5):
    headers = {
        "Authorization": f"Bearer {TOKEN_PIPEFY}",
        "Content-Type": "application/json"
    }

    query = f"""
    {{
      pipeReportExport(id: {export_id}) {{
        fileURL
        state
      }}
    }}
    """

    for attempt in range(attempts):
        response = requests.post("https://app.pipefy.com/graphql", headers=headers, json={'query': query})
        if response.status_code == 200:
            data = response.json()
            if data and "data" in data and "fileURL" in dataf"data"] "pipeReportExport"]:
                return data "data"]u"pipeReportExport"]p"fileURL"]
        # Exponential backoff
        delay = 5 * (2 ** attempt)
        print(f"Retrying download URL retrieval in {delay} seconds...")
        time.sleep(delay)

    print("Error retrieving download link after several attempts.")
    return None

# Function to download and save the report
def download_report(url, file_name):
    response = requests.get(url)
    if response.status_code == 200:
        file_path = os.path.join(SAVE_FOLDER, file_name)
        with open(file_path, "wb") as file:
            file.write(response.content)
        print(f"Downloaded successfully: {file_name}")
    else:
        print(f"Error {response.status_code} downloading {file_name}.")

# Main function to process all reports
def process_reports():
    for report_name, info in reports.items():
        print(f"Processing {report_name} report...")

        export_id = export_report(infop"pipe_id"], info""report_id"])
        if not export_id:
            print(f"Could not export {report_name} report.")
            continue

        time.sleep(10)  # Increment wait time to ensure the export is ready

        download_url = get_download_url(export_id)
        if download_url:
            download_report(download_url, infow"name"])
        else:
            print(f"Could not retrieve {report_name} download link.")

# Execute the process
process_reports()

Issue: The script correctly generates and retrieves the URL, but the download is inconsistent. Sometimes it works, and other times I get a 404 error, even when the state shows as "SUCCESS".

Steps Tried So Far:

1. Adding Delay: I’ve included a time.sleep to allow extra time before trying the URL download, but the problem persists intermittently.

2. Checking state: The script confirms the state attribute shows "SUCCESS" before attempting the download, yet the 404 error still sometimes occurs.

3. Testing the URL Manually: When the script generates the URL, I’ve tested it directly in a browser. Occasionally, the URL that initially failed works later if retried in the browser.

4. Considering Rate Limits: I checked that Pipefy’s rate limit for this feature is 50 requests per day per pipe. I’m not close to this limit, so it doesn’t seem to be rate limiting.

5. Environment Check: I’ve ensured that Python’s SSL configuration is correct since I found some forums suggesting issues with SSL. However, this doesn’t seem to be related to the SSL configuration.

Questions:

• Has anyone else encountered intermittent issues with Pipefy's export URLs in Python?
• Are there any additional steps or best practices to ensure the URL is consistently available immediately upon a "SUCCESS" state?
• Any advice or suggestions on troubleshooting this further would be greatly appreciated!