🖨️
DataSF | Publishing Process
  • Introduction
  • Why Publish Data?
    • Publishing Data Standards
  • Publishing Specifications
    • Kickoff
      • Breadcrumbs and Inventory
    • Privacy Toolkit
      • Privacy Toolkit Form
    • Security Toolkit
  • Data Pipeline
    • Pipeline Basics
      • Manual Publishing
      • Data Pipeline
      • Common Transformations
  • Metadata
    • Intro to Metadata
      • Metadata Standards
  • Publishing & Maintenance
    • Review & Publishing
    • See our other explainers
Powered by GitBook
On this page
  • Adding Supervisor District or Analysis Neighborhood
  • Adding a "data_published_at" field
  1. Data Pipeline
  2. Pipeline Basics

Common Transformations

PreviousData PipelineNextIntro to Metadata

Last updated 1 year ago

Adding Supervisor District or Analysis Neighborhood

The transformation can be done after the data is loaded into the open data portal utilizing the . You just have to add a new column and apply this formula to it:

  • Start on the dataset's information page ()

  • If you have edit permissions, click on on "Edit" in the upper right corner

  • Click "Review Data" in the Review and Configure Data box

  • Click "+Add Column" on the left-hand bar

  • Add a name and description, and add "No Source" in the Source Column dropdown (this will create a blank column), then add a data type and click "Done"

  • Back in the Review Data tab, scroll to your new column, click the three dots next to the column name, and select "Data Transforms"

  • Using the formula box, type in the Region Code Function (snippet below), test it by hitting "run"

forgive(region_code_label(<sourcecolumn>,'{4X4 of the lookup dataset}','data_column'))
  • After verifying it is running, return to the draft homepage and click "Update"

Current Supervisor District:

  • forgive(region_code_label(<point_column>,'cqbw-m5m3','sup_dist'))

Analysis Neighborhood:

  • forgive(region_code_label(<point_column>,'sevw-6tgi','neighborhoods_analysis_boundaries'))

Adding a "data_published_at" field

This can also be done within socrata after the dataset is loaded using the same process as region code transforms (above), but using a different code snipped in the data transformation box:

to_floating_timestamp(source_created_at(), 'US/Pacific')

Region Code Function
example