Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Log into Snowflake using this URL: https://snaplogic.snowflakecomputing.com/console

  2. Create a Snowflake Storage Integration Object. For example,
    create storage integration JOHN_GCS_STORAGE_INTEGRATION1

      type = external_stage

      storage_provider = gcs

      enabled = true

      storage_allowed_locations = ('gcs://johnsnowflake1/', 'gcs://johnsnowflake2/');

  3. Describe the Snowflake storage integration object to get the GCP Service Account.
    desc integration JOHN_GCS_STORAGE_INTEGRATION;

  4. Create a Custom Role.

    1. Navigate to IAM & Admin > Roles > CREATE ROLE

    2. Create a role with the an appropriate name and add the following permissions:

      • firebase.projects.get

      • resourcemanager.projects.get

      • storage.buckets.create

      • storage.buckets.delete

      • storage.buckets.get

      • storage.buckets.getIamPolicy

      • storage.buckets.list

      • storage.buckets.setIamPolicy

      • storage.buckets.update

      • storage.objects.create

      • storage.objects.delete

      • storage.objects.get

      • storage.objects.getIamPolicy

      • storage.objects.list

      • storage.objects.setIamPolicy

      • storage.objects.update

  5. Provide permission on the Google Cloud Storage bucket to the Snowflake GCP Service Account.

  6. Add the GCP service user and the custom role you have just created.

  7. Bulk load the data to Snowflake.
    COPY INTO "PUBLIC".JOHN_EMP2 

    FROM 'gcs://johnsnowflake1/data/' 

    FILES = ( 'john_emp1.csv' )  

    FILE_FORMAT = (TYPE = CSV SKIP_HEADER = 1)

    STORAGE_INTEGRATION = JOHN_GCS_STORAGE_INTEGRATION;

  8. Unload the data from Snowflake table to a GCS location:

    copy into 'gcs://johnsnowflake1/unload/'

    from "PUBLIC".JOHN_EMP2

    storage_integration = JOHN_GCS_STORAGE_INTEGRATION

    file_format = (format_name = snap_csv_format);

...

...