> For the complete documentation index, see [llms.txt](https://help.citrusad.com/retail-media-interface/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.citrusad.com/retail-media-interface/integration/data-api/data-overview-1/syncing-data-via-file-protocols-1.md). # Sync data via file protocols This section explains the protocols supported by Epsilon Retail Media and provides guidelines for naming files to streamline automatic downloads from your server. For secure downloads, send your Epsilon Retail Media representative the necessary details—protocol type, host, port, file path—and any required authentication credentials, such as a username and password. You must store your data files on the server to enable retrieval through standard protocols, such as the Google Cloud Storage (GCS), HTTPS, and SFTP. ## Supported protocols Here are the supported protocols that offer secure methods for file storage and transfer, ensuring safe communication with Epsilon Retail Media systems. ### GCS (recommended) GCS is our preferred mechanism for file storage and retrieval due to its robust security features and seamless integration with Epsilon Retail Media systems. ### HTTPS The HTTPS protocol provides a secure method for file transfer by supporting password-based authentication. This ensures encrypted communication between your server and Epsilon Retail Media. {% hint style="warning" %} Larger files are better synchronised via GCS Feeds over 100,000 products are best synchronised by bucket methodology to improve performance. {% endhint %} ### SFTP When using the Secure File Transfer Protocol (SFTP), Epsilon Retail Media supports two methods of authentication for downloading data files: * **Username and password**: This standard method uses credentials to authenticate access. * **Public key authentication**: You can also use Epsilon Retail Media's public key. For this, ensure that the public key provided below is installed on your SFTP server to facilitate secure file transfers. #### Staging ``` ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDMd4w5Z6hsjlUc0/r3dTJ+IaM9Mm+l3ZTrX3D5+30ZNiAb3Mhvlp8gv37CFKzqM90oFejl15bbr5dymjycsOWInxqkMyLOTbdMFAk7hmicNvN5bt3tx/Z0UewNvAx0Uw9Qxu7SCMSAalk9l7dqLp5RFFj+RHq9ApV8KIA/Fw2mjCw2vrJhiRWeuUuNNQTbGFBA5y72ATeYA2p0ZK3/TTJxyxCL70zGVxFG0jdAurfXKujYaF73Ql/gl5wmtnxGvTORtvObQdc2jCTsynj1w17LSZAdeoPQGnQkVCk37ZUnWTw8s4RYjHFVunsTDxJO5Qnw5yj8oqhe5IdpQyavUj+otRd6900q/3dK60IELH6AFz+ZEqudlBh8ZO2hvHIDS2Ii5S3LunlejykAVdRl7RVHA17Pk3mmrHCS6i3n9ZliBYeVQiKme2jHVB+ESrHFxQWl3ct9MYFNlAqwuhTFQRSG9ioSEHX5Dnmqk8aqbm/4pYY4b7YHOcIpHrtrVrFTJ3vLM1If1OVrDQidFhi1ArmoCU+PO65jNSeMj27m/HoX7Mbcmi8F++Op3MgP8iJrohE0Wjz+use7nlDtQTtDxDeEQQYGqsz4DOtH/jn1TWRynaXZqwPqC5Q6b52QM22T+E3rgT/4G7yJU3IRKioByOck251MEtOEgSCvvIAaV685fQ== ``` #### Production ``` ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCtpIUFoYu8OUesZGP9rffv6/hky1HbQCAhC8hNRmcWpdptzRYsKIZFPz9t2vb85dierTZqJstAEJkePHd25cZ8btIBacbAR49avCIXiBuVgvlJk1+Er2dyYqSzSeFhpzlZxi4CkGo/Nsx7yxPi5693fdVF3vr+/iencw1tdLJJXfT6j1YLBCREg+IziBCO6m9P0QQGEDXYvsOmIzl0CJ8enU7gjvnYL5ify1/aGpoXv9/XcCYJWXTQmvmISknOMdC4pBiuDqsE9CX27FLQQb1ZT5jNrqD5hFsEo0rWqLiCbb5qoxIdMSe/IbA+sewgpgTPNJXeCNLSSH/MHhj2IgiMZGDDDaVaqMZD7N2jeQdI6SQbgvXiLTTUL4gNyO5eN9oqoeJumeZMi0+MQ7kkfHDYGtKe3RMxL7heL650HL+eWDIybezyPdQmfyVew9KuN0uB+2X+2NxQs8ZLqow486XDfUDfo+rPuoIvkYy7Z8ox+gne+1iAk43TCekcDwxlc56R9muMtGLuzzg1x9s9MqsAg5JNWEPQ8giAwCQJwm7h3UcCY3TJCn0LUzYHc7Ycr11JfIXL47vYGH3C0XBYlbEU/8m4VhhexzgJi8G/reclY5pfjtwQGOiobsxLsWgGx6Qoz/qWqmGwQIg+qi7jvA7jnltUP+RTFZpmd18/dSxB+w== ``` ## Data compression and encryption Before uploading data to your server for syncing, you may need to compress and encrypt the files. When both compression and encryption are used on data files, Epsilon Retail Media assumes compression occurs before encryption. * For data compression, we support zip and gzip formats. However, we recommend gzip over zip due to its better compression ratios and widespread support. Compressed files are decompressed before processing. * For data encryption, files are decrypted upon download. Use the provided public keys to encrypt files before uploading to your server, ensuring only Epsilon Retail Media can access the data securely. ## File naming convention To enable Epsilon Retail Media to download and process your data file, you need to provide the protocol, host, port, and file path details. Choose a convenient daily time to ensure your data file is ready on the server, as Epsilon Retail Media will download it each day. The most straightforward approach to specify a daily target file for Epsilon Retail Media's downloads is by using an explicit file name. This allows Epsilon Retail Media to directly retrieve the file from your server. For those using FTP, FTPS, or SFTP protocols, we also offer additional methods for specifying target files through what we call target file modes. These options include: * Rolling\_earliest * Rolling\_earliest\_24\_hours * Rolling\_latest * Rolling\_latest\_24\_hours. Selecting a target file mode requires you to provide a text-based template for data file names. This template includes a special string, `{\*}`, which helps Epsilon Retail Media match file names on your server and select the correct file each day. For example, consider the template `ERMCatalogData_AU_{\*}.txt`. This template requires that file names begin with `ERMCatalogData_AU` and end with `.txt`. Using this template, the following file names would match: `ERMCatalogProduct_AU_20190315.txt` `ERMCatalogProduct_AU_20190314.txt` `ERMCatalogProduct_AU_20190312.txt` To avoid downloading data files that are still in the process of being uploaded, Epsilon Retail Media only downloads files that were last modified more than one minute before the server is accessed. Although multiple files might match the template, Epsilon Retail Media processes only one file at a time. Selecting the appropriate file from multiple candidates is guided by the chosen target file mode, which is discussed in detail below. ### Rolling\_earliest In this target file mode, files are filtered based on the provided name template. Once filtered, the files are sorted by ascending order of their names, and the first file in this order is selected for download. For example, if the file name template is `ERMCatalogData_AU_{\*}.txt` and the filtered list of file names includes the following: `ERMCatalogProduct_AU_20190312.txt` `ERMCatalogProduct_AU_20190313.txt` `ERMCatalogProduct_AU_20190314.txt` In this mode, then the file `ERMCatalogProduct_AU_20190312.txt` is chosen for download. ### Rolling\_earliest\_24\_hours In this target file mode, files are initially filtered based on the specified name template. From the filtered list, only files modified within the past 24 hours are considered. The files are then sorted in ascending order by name, and the first file in this order is selected for download. For example, let's assume the current time is `2019-03-15 10:30:07`, and the file name template is `ERMCatalogData_AU_{\*}.txt`. If the filtered list of file names includes the following: | File name | Last modified | | ----------------------------------- | ----------------- | | ERMCatalogProduct\_AU\_20190312.txt | 13/03/19 15:35:11 | | ERMCatalogProduct\_AU\_20190313.txt | 13/03/19 15:35:08 | | ERMCatalogProduct\_AU\_20190314.txt | 14/03/19 15:35:10 | | ERMCatalogProduct\_AU\_20190315.txt | 15/03/19 10:05:07 | In this mode, then the file `ERMCatalogProduct_AU_20190314.txt` is chosen for download. ### Rolling\_latest In this target file mode, files are filtered based on the specified name template. After filtering, the files are sorted in descending order by their names. The first file in this sorted list is selected for download. For example, if the file name template is `ERMCatalogData_AU_{\*}.txt`, and the filtered list of file names includes the following: `ERMCatalogProduct_AU_20190314.txt` `ERMCatalogProduct_AU_20190313.txt` `ERMCatalogProduct_AU_20190312.txt` In this mode, then the file `ERMCatalogProduct_AU_20190314.txt` is chosen for download. This mode is similar to Rolling\_earliest, but instead of sorting the files in ascending order, they are sorted in descending order. ### Rolling\_latest\_24\_hours In this target file mode, files are initially filtered using the specified name template. From the filtered list, only files modified within the past 24 hours are considered. The files are then sorted in descending order by their names, and the first file in this order is selected for download. For example, assume the current time is `2019-03-15 10:30:07`, and the file name template is `ERMCatalogData_AU_{\*}.txt`. If the filtered list of file names includes: | File name | Last modified | | ----------------------------------- | ----------------- | | ERMCatalogProduct\_AU\_20190312.txt | 13/03/19 15:35:11 | | ERMCatalogProduct\_AU\_20190313.txt | 13/03/19 15:35:08 | | ERMCatalogProduct\_AU\_20190314.txt | 14/03/19 15:35:10 | | ERMCatalogProduct\_AU\_20190315.txt | 15/03/19 10:05:07 | In this mode, then the file `ERMCatalogProduct_AU_20190315.txt` is chosen for download. This mode is similar to Rolling\_earliest\_24\_hours, but here files are sorted in descending order rather than ascending. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://help.citrusad.com/retail-media-interface/integration/data-api/data-overview-1/syncing-data-via-file-protocols-1.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.