Skip to main content

Amazon PostgreSQL

This guide contains information required in setting up an Amazon PostgreSQL Source in Vanus Connect.

Introduction​

PostqreSQL​

PostgreSQL is a reliable and feature-rich open-source database system with advanced features like multi-version concurrency control(MVCC), full-text search, and geospatial data management. It's commonly used for data-intensive applications and web applications due to its scalability and robustness. PostgreSQL works on various operating systems and provides both command-line and graphical user interface tools for data management and application development.

AWS-PostgreSQL​

AWS PostgreSQL is a managed database service offered by Amazon Web Services (AWS) that provides a fully-managed, scalable, and secure PostgreSQL database in the cloud. It simplifies tasks like database backups, software patching, and replication, allowing users to focus on their application development and business needs. AWS PostgreSQL features high availability, automated backups and recovery, automatic failover, and advanced security features like encryption. It also supports popular PostgreSQL extensions like PostGIS and pgAdmin. Overall, AWS PostgreSQL is a reliable option for running PostgreSQL databases in the cloud with flexibility and scalability.

PostgreSQL CDC​

PostgreSQL CDC (Change Data Capture) is a feature that captures and tracks real-time changes made to a PostgreSQL database. It works by monitoring the database's transaction log and capturing any detected changes, which are stored separately for further analysis or processing. CDC is useful for real-time data integration and synchronization in scenarios such as data warehousing, business intelligence, or ETL processes, ensuring that downstream systems have access to the latest information and data is synchronized across different systems in real-time.

PostgreSQL CDC Source​

The Vanus PostgreSQL Source connector is a connector provided by Vanus, that identifies and tracks data changes from a PostgreSQL database in real-time. It extracts this data from the users postgresql database, converts it to cloud events and streams it to a sink connector of their choice. CDC is a technique used to identify and track changes made to a database, such as insertions, updates, and deletions.

Prerequisites​

Before creating your PostgreSQL connection, you must have:

Getting Started​

Perform the following steps to configure your Amazon RDS PostgreSQL Source.

Step 1: Set Up a PostgreSQL Server in AWS Using RDS (Optional)​

  1. Log in to the AWS Management Console using your root account credentials.

  2. Type RDS① in the search box and select RDS② from the results.

aws-postgres-source-1

  1. Click the Region name③ to open the dropdown menu and select the particular Region④ in which you want to create the DB instance.

aws-postgres-source-2

  1. Click Create database⑤.

aws-postgres-source-3

  1. Select Standard create⑥ and choose the PostgreSQL⑦ icon.

aws-postgres-source-4

  1. Select the Free tier⑧ Template for testing, or opt for one of the other available options based on your specific use case.

aws-postgres-source-5

note

Multi-AZ DB⑨ deployment is only available for the paid tier. A Multi-AZ deployment automatically provisions and maintains a synchronous standby replica in a different Availability Zone.

  1. Enter the following information.

  • DB instance identifier⑩: Give a unique name to your instance.

  • Master username⑪: This is the username that will be used to log in to your DB instance.

  • Master password⑫: This is the password that will be used to log in with the username.

  • Confirm master password⑬: Write the password again.

aws-postgres-source-6

  1. Select the General Purpose SSD (gp2)⑭ storage type. It's efficient and cost-effective for our intended use.

aws-postgres-source-7

  1. Enable Public access⑮.

aws-postgres-source-8

note

This allocates an IP to your database instance and allows direct connection to the database from the internet.

  1. Select Create new⑯ to create a new VPC security group, then specify your New VPC security group name⑰.

aws-postgres-source-9

  1. Select Password authentication⑱.

aws-postgres-source-10

note

Leave Performance Insights⑲ checked, or uncheck it if you have no use for it.

  1. Input an Initial database name⑳, and set your preferred Backup retention period㉑.

aws-postgres-source-11

  1. Click the Create database㉒ button to complete the process of creating your database.

aws-postgres-source-12

Step 2: Amazon RDS PostgreSQL Connection Settings​

To set up Amazon RDS PostgreSQL Source in Vanus Connect:

  1. Return to the RDS page and click on the database① you just created.

aws-postgres-source-13

  1. Copy your Endpoint & port②.

aws-postgres-source-14

  1. Navigate to the Vanus Cloud platform and fill in the following credentials.
  • Host③: This is your RDS endpoint.
  • Port④: Enter the port number on which your PostgreSQL server is listening. The default port for PostgreSQL is 5432.
  • Username⑤: Master username selected during the database creation.
  • Password⑥: User password.
  • Database⑦: The initial database name.
  • Schemas⑧: PostgreSQL schema
  • Tables⑨: The name of the table you want your events to be sent to.
  • Click Next⑩ and finish the configurations.

aws-postgres-source-15

Learn more about Vanus and Vanus Connect in our documentation.