At Switchboard, we imagine the data stack as a wedding cake. The top tier comprises data visualization dashboards (Tableau, Looker, Power BI etc), as well as custom applications and data science projects.

The second tier typically comprises the data lake (or warehouse), which stores the data. These are increasingly cloud-based, such as Snowflake, BigQuery, AWS or Redshift, although customers also use storage facilities such as GCS buckets with Parquet files; HDFS (Hadoop Distributed File System); or SFTP.

The third tier — where the Switchboard application sits — is what many of our customers call the ‘partner ecosystem' or ‘data ecosystem'. We call it the Data Operations (DataOps) layer — the single pane of glass that sits on top of a messy ecosystem of third-party data, APIs, JSON objects, CSVs, databases, or email. Switchboard's cloud-hosted platform provides a consistent and repeatable set of workflows and processes to bring together all this data through default standard connectors, aggregate and normalize it using a transparent technology and workflow, and load it into the EDW.

First, we extract very granular, large scale sets of raw data, for instance from Google Campaign Manager (GCM), which can be used for ad hoc queries according to a host of different metrics such as country, advertising partner, etc. We also store all the metadata necessary to understand the mechanics of data access. This includes the time we access the report, the download start and finish time, and any errors (such as a dirty row that doesn’t match the expected schema) so that business teams can account for any outages or glitches they encounter.

Second, we create an aggregation of that ginormous set of raw data to produce bespoke tables for use by the business teams, for instance, a conversion report, or any other slice of the data required.

For most marketing sources, we pull the data using proprietary connectors that access their reporting APIs (although in some cases, the platform provides raw files). We usually provide a daily report using data from the previous day. This is the cadence at which platforms such as Facebook, Snapchat, TikTok or Twitter can report accurately. Marketing data often changes over time as conversions can happen days after a campaign is viewed, so Switchboard also provides the capability to re-ingest reports over time.

The whole point of aggregating and normalizing data is so that it can be used downstream. So, whether you need a Gzip csv; or Parquet, Spark, or Avro files; or tables in a cloud data warehouse such as BigQuery, we can send the data across in a compatible format.

Switchboard’s enterprise-level technology provides automated scale, automated monitoring alerts, and a full set of logging and auditing capabilities. Even for those organizations with sophisticated DataOps infrastructures in place, the process of manually querying the data each time a business team needs a specific report becomes unsustainable – not to mention pulling engineering teams away from other business-critical initiatives.

With Switchboard, non-technical business teams can make quick iterations in a few clicks and generate different versions of reports in minutes – and they can quickly move data to other downstream systems whenever they need to.

It's all automated in Switchboard. So as a configuration, you can schedule a “30-day lookback”, where that past 30 days of data is automatically refreshed every day. And you can set these lookbacks across the hundreds of data sources that we support right now. So, any net new request is just as simple as publishing a new “recipe” (normalization template).

Yes, we do all of that for you within the Switchboard application. Certain platforms are notorious for rapid API changes, so it’s our job to keep on top of these changes and ensure that APIs are always up to date.

It depends on the data source and connector, but when we can, we do so within Switchboard’s script editor. For example, when an API changes its dimension and metric names, or when an adtech API has versions that are mutually exclusive, we have different versions of data connectors for those APIs so we can still pull the data as needed.

Yes, Switchboard supports these types of joins. If you use data transfer files with GAM, we often do joins of the ad units, line items, orders, and creatives, because those files only have IDs, whereas business teams often just want to use the names or custom fields from the line item, for instance.

Yes, each time you create a new draft (or ‘branch’) you can play around and add different test parameters, much like a sandbox. The Switchboard Compiler will automatically provide error messages alerting you to any invalid dimensions or metrics – all before you publish.

Yes, every new branch is tagged with a person and a timestamp. If there is a conflict, the Switchboard application will automatically tell the user before they can publish.

Using Switchboard’s backfill function, teams can go back and automatically pull specific periods of the data – for example, the past month – at the click of a button. This is particularly useful for business teams who quickly decide they need to see a different period of the data, as they can simply change the recipe and pull a different period without burdening their engineering team.