Skip to content

Commit 5ad1ced

Browse files
author
王平10304955
committed
Add SFTP support to gpfdist for data ingestion
Feature:Supporting the Loading of bz(bz2) Format Files gpfdist, Cloudberry's parallel file distribution program, traditionally required data files to be co-located with the gpfdist process. This limitation made it cumbersome to load data from remote servers, often requiring an extra data transfer step. This commit extends gpfdist to support the SFTP protocol, enabling users to ingest data directly from remote servers. This enhancement streamlines ETL workflows by allowing `CREATE EXTERNAL TABLE` to specify SFTP locations. Key change information: Implement the loading of .bz2 files by utilizing the read functions provided by the libssh2 library.
1 parent b4cc332 commit 5ad1ced

File tree

1 file changed

+1
-1
lines changed
  • src/backend/utils/misc/fstream

1 file changed

+1
-1
lines changed

src/backend/utils/misc/fstream/gfile.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,7 @@ bz_file_read(gfile_t *fd, void *ptr, size_t len)
244244
if (fd->is_sftp)
245245
{
246246
#ifdef LIBSSH2
247-
gfile_printf_then_putc_newline("sftp_rean : Read bz files from an SFTP server");
247+
gfile_printf_then_putc_newline("sftp_read : Read bz files from an SFTP server");
248248
s = sftp_read(fd, z->in + z->in_size, sizeof z->in - z->in_size);
249249
#endif
250250
}

0 commit comments

Comments
 (0)