Skip to main content

Slack

Availability:
Available Airbyte CloudAvailable Airbyte OSSAvailable PyAirbyte
Support Level:
Airbyte Connector
Connector Version:
1.3.1(Last updated 5 days ago)
CDK Version:
3.5.1
Sync Success Rate:
Usage Rate:

This page contains the setup guide and reference information for the Slack source connector.

Prerequisites​

OAuth or API Token (via Slack App or Legacy API Key) is required for access to Slack.

You can no longer create "Legacy" API Keys, but if you already have one, you can use it with this source. Fill it into the API key section.

We recommend creating a restricted, read-only key specifically for Airbyte access. This will allow you to control which resources Airbyte should be able to access.

Note that refresh token are entirely optional for Slack and are not required to use Airbyte. You can learn more about refresh tokens here.

Setup guide​

Step 1: Set up Slack​

info

If you are using an "legacy" Slack API, skip to the Airbyte Open Source additional setup steps section below.

In order to pull data out of your Slack instance, you need to create a Slack App. This may sound daunting, but it is actually pretty straight forward. Slack supplies documentation on how to build apps. Feel free to follow that if you want to do something fancy. We'll describe the steps we followed to creat the Slack App for this tutorial.

info

This tutorial assumes that you are an administrator on your slack instance. If you are not, you will need to coordinate with your administrator on the steps that require setting permissions for your app.

  1. Go to the apps page

  2. Click "Create New App"

  3. It will request a name and the slack instance you want to create the app for. Make sure you select the instance form which you want to pull data.

  4. Completing that form will take you to the "Basic Information" page for your app.

  5. Now we need to grant the correct permissions to the app. (This is the part that requires you to be an administrator). Go to "Permissions". Then under "Bot Token Scopes" click on "Add an OAuth Scope". We will now need to add the following scopes:

     channels:history
     channels:join
     channels:read
     files:read
     groups:read
     links:read
     reactions:read
     remote_files:read
     team:read
     usergroups:read
     users.profile:read
     users:read

    This may look daunting, but the search functionality in the dropdown should make this part go pretty quick.

  6. Scroll to the top of the page and click "Install to Workspace". This will generate a "Bot User OAuth Access Token". We will need this in a moment.

  7. Now go to your slack instance. For any public channel go to info => more => add apps. In the search bar search for the name of your app. (If using the desktop version of slack, you may need to restart Slack for it to pick up the new app). Airbyte will only replicate messages from channels that the Slack bot has been added to.

  8. In Airbyte, create a Slack source. The "Bot User OAuth Access Token" from the earlier should be used as the token.

  9. You can now pull data from your slack instance!

Airbyte Open Source additional setup steps

You can no longer create "Legacy" API Keys, but if you already have one, you can use it with this source. Fill it into the API key section.

We recommend creating a restricted, read-only key specifically for Airbyte access. This will allow you to control which resources Airbyte should be able to access.

Step 2: Set up the Slack connector in Airbyte​

For Airbyte Cloud:

  1. Log into your Airbyte Cloud account.
  2. In the left navigation bar, click Sources. In the top-right corner, click +new source.
  3. On the Set up the source page, enter the name for the Slack connector and select Slack from the Source type dropdown.
  4. Select Authenticate your account and log in and Authorize to the Slack account.
  5. Required Enter your start_date.
  6. Required Enter your lookback_window, which corresponds to amount of days in the past from which you want to sync data.
  7. Toggle join_channels, if you want to join all public channels or to sync data only from channels the bot is already in. If not set, you'll need to manually add the bot to all the channels from which you'd like to sync messages.
  8. Enter your channel_filter, this should be list of channel names (without leading '#' char) that limits the channels from which you'd like to sync. If no channels are specified, Airbyte will replicate all data.
  9. Toggle include_private_channels if you want to sync data from private channels. You will need to manually add the bot to private channels, join_channel does not work with private channels.
  10. Click Set up source.

For Airbyte Open Source:

  1. Navigate to the Airbyte Open Source dashboard.
  2. Set the name for your source.
  3. Required Enter your start_date.
  4. Required Enter your lookback_window, which corresponds to amount of days in the past from which you want to sync data.
  5. Toggle join_channels, if you want to join all channels or to sync data only from channels the bot is already in. If not set, you'll need to manually add the bot to all the channels from which you'd like to sync messages.
  6. Enter your channel_filter, this should be list of channel names (without leading '#' char) that limits the channels from which you'd like to sync. If no channels are specified, Airbyte will replicate all data.
  7. Toggle include_private_channels if you want to sync data from private channels. You will need to manually add the bot to private channels, join_channel does not work with private channels.
  8. Enter your api_token.
  9. Click Set up source.

Supported sync modes​

The Slack source connector supports the following sync modes:

FeatureSupported?
Full Refresh SyncYes
Incremental SyncYes
NamespacesNo

Supported Streams​

For most of the streams, the Slack source connector uses the Conversations API under the hood.

Performance considerations​

The connector is restricted by normal Slack requests limitation.

It is recommended to sync required channels only, this can be done by specifying config variable channel_filter in settings.

Data type map​

Integration TypeAirbyte Type
stringstring
numbernumber
arrayarray
objectobject

Limitations & Troubleshooting​

Expand to see details about Slack connector limitations and troubleshooting.

Connector limitations​

Rate limiting​

Slack has rate limit restrictions.

Troubleshooting​

  • Check out common troubleshooting issues for the Slack source connector on our Airbyte Forum here.

Reference​

Config fields reference

Field
Type
Property name
string
start_date
integer
lookback_window
boolean
join_channels
boolean
include_private_channels
array<string>
channel_filter
object
credentials

Changelog​

Expand to review
VersionDatePull RequestSubject
1.3.12024-07-2442485Fix MRO error for IncrementalMessageStream
1.3.02024-07-1741994Migrate to CDK v3.5.1
1.2.02024-07-1641970Migrate to CDK v2.4.0
1.1.132024-07-1341863Update dependencies
1.1.122024-07-1041485Update dependencies
1.1.112024-07-0941231Update dependencies
1.1.102024-07-0640839Update dependencies
1.1.92024-06-2540347Update dependencies
1.1.82024-06-2240166Update dependencies
1.1.72025-06-1439343Update state handling for threads Python stream
1.1.62024-06-1239132Respect include_private_channels option in threads stream
1.1.52024-06-1039132Convert string state to float for threads stream
1.1.42024-06-0639271[autopull] Upgrade base image to v1.2.2
1.1.32024-06-0539121Change cursor format for channel_messages stream to %s_as_float
1.1.22024-05-2338619Fix cursor granularity for the channel_messages stream
1.1.12024-05-0236661Schema descriptions
1.1.02024-04-1837332Add the capability to sync from private channels
1.0.02024-04-0235477Migration to low-code CDK
0.4.12024-03-2736579Upgrade airbyte-cdk version to emit record counts as floats
0.4.02024-03-1936267Pin airbyte-cdk version to ^0
0.3.92024-02-1235157Manage dependencies with Poetry
0.3.82024-02-0935131Fixed the issue when schema discovery fails with 502 due to the platform timeout
0.3.72024-01-101234Prepare for airbyte-lib
0.3.62023-11-2132707Threads: do not use client-side record filtering
0.3.52023-10-1931599Base image migration: remove Dockerfile and use the python-connector-base image
0.3.42023-10-0631134Update CDK and remove non iterable return from records
0.3.32023-09-2830580Add bot_id field to threads schema
0.3.22023-09-2030613Set default value for channel_filters during discover
0.3.12023-09-1930570Use default availability strategy
0.3.02023-09-1830521Add unexpected fields to streams channel_messages, channels, threads, users
0.2.02023-05-2426497Fixed lookback window value limitations
0.1.262023-05-1726186Limited the lookback window range for input configuration
0.1.252023-03-2022889Specified date formatting in specification
0.1.242023-03-2024126Increase page size to 1000
0.1.232023-02-2121907Do not join channels that not gonna be synced
0.1.222023-01-2722022Set AvailabilityStrategy for streams explicitly to None
0.1.212023-01-1221321Retry Timeout error
0.1.202022-12-2120767Update schema
0.1.192022-12-0119970Remove OAuth2.0 broken refresh_token support
0.1.182022-09-2817315Always install latest version of Airbyte CDK
0.1.172022-08-2816085Increase unit test coverage
0.1.162022-08-2816050Fix SATs
0.1.152022-03-3111613Add 'channel_filter' config and improve performance
0.1.142022-01-269575Correct schema
0.1.132021-11-087499Remove base-python dependencies
0.1.122021-10-076570Implement OAuth support with OAuth authenticator
0.1.112021-08-275830Fix sync operations hang forever issue
0.1.102021-08-275697Fix max retries issue
0.1.92021-07-204860Fix reading threads issue
0.1.82021-07-144683Add float_ts primary key
0.1.72021-06-253978Release Slack CDK Connector
Was this page helpful?