Dogfood Perforce server

Perforce is a version control system like Git, subversion, or mercurial. While git is based on a distributed, decentralised model, Perforce is centralised. For testing purposes, you may use our Perforce dogfood server.

Setting up

  • To connect to the Perforce server, you’ll need the Perforce cli installed locally. Use the command: brew install --cask perforce

  • The following environment variables configure your shell to point at the Perforce server. Set them to your env with the export command, or add them to your .bashrc or .zshrc file. 

    # .zshrc or .bashrc

    export P4PORT=perforce.sgdev.org:1666 # This points the p4 cli at the dogfood server
    export P4USER=admin                   # sets your user
    export P4PASSWD=<session ticket>      # optional, see details below, doesn't require string quotes
    export P4EDITOR=/usr/bin/vim          # specifies the editor opened by some p4 commands

Perforce dogfood is a service on our Sourcegraph dogfood cluster, for more info see its GCP service details.

Interacting with Perforce dogfood

To add repos to the Perforce dogfood server follow the following procedure:

Generate a session ticket

Interaction with the dogfood server requires authentication. Once you’ve set up your shell environment you’ll need to generate a session ticket with your users password. You can find the admin password, and others on our shared 1Password account.

  1. Once your environment is set run p4 ping this will prompt you for the admin password, and is a good way to test your connection to the server.
  2. With your connection confirmed generate a session ticket with the following command: p4 -u <p4.user> login -p -a you’ll be prompted for your user password, once you’ve entered it a session ticket will be printed to STDOUT. You may also run p4 -u <p4.user> login -a to set the ticket in ~/.p4tickets, this allows you to skip setting P4PASSWD in your env, and will only work if you have no P4PASSWD variable in your env.
  3. Set the ticket generated in step 2 to your P4PASSWD env variable
  4. p4 commands should no longer require a password. Note this ticket expires every 12 hours unless configured to do otherwise.

Create new depot

Perforce uses depots rather than repos, the concept is functionally equivalent. To see all the repos on the perforce server run p4 depots, to create a new depot on the server run p4 depot <depot name>, this will open a file like seen below:

Depot:  support-tools

Owner:  admin

Date:   2021/08/08 

Description:
        testing testing testing

Type:   local

Address:        local

Suffix: .p4s

StreamDepth:    //support-tools/1

Map:    support-tools/...

Once a depot is created on the server, we can start to add files from our local client to it.

While this new depot has been created, you still need to add it to “client/workspace” and have perforce track it. The easiest way to do this is in the p4v UI. Go to Connection > Edit Current Workspace > Right-click on the depot you just created > Include Tree.

You may see a warning about no files. Create a basic text file then run: p4 add init-file.txt then run p4 submit -d "initial commit".

Create a client (workspace)

Perforce is different than git in that it utilizes a concept called clients (or more recently, workspaces). This is a subset of files on your machine that mirrors files on the Perforce server. The p4 client <name> command will open a client spec file with an editor specified by the P4EDITOR env variable. Below is an example:

Client: warren

Update: 2021/08/08 

Access: 2021/08/08 

Owner:  admin

Host:   Warrens-MacBook-Pro.local

Description:
        Created by admin.

Root:   /Users/warrengifford

Options:        noallwrite noclobber nocompress unlocked nomodtime normdir

SubmitOptions:  submitunchanged

LineEnd:        local

View:
        //test-large-depot/... //warren/test-large-depot/...
        //test-perms/... //warren/test-perms/...
        //spec/... //warren/spec/...
        //depot/... //warren/depot/...
        //Tone.js/... //warren/Tone.js/...
        //LifeBox/... //warren/LifeBox/...

You’ll notice under View: a “Perforce path” or depot path on the left, followed by a client path on the right. The depot path reflects the path to the depot on the Perforce server, while the client path is the path to the directory on your local machine that you want to map to the Perforce depot.

Specifying views in your client configuration allows you to declare which files from the Perforce depot are relevant to you, or which files you want to be part of your workspace. You can learn more about this topic here.

Once you’ve created a depot on dogfood, map a local directory to it by adding an entry to your client spec under Views:

//<depot name>/... //<client name>/<root directory>/...

Note that the client name is mapped to your Root: setting in the client spec.

Add files and submit

Once you’ve created a depot on the server, and created a client spec, adding files is a lot like git. To add the files run p4 add <relative path to files>/... for example while in /Users/warrengifford I could run p4 add ./LifeBox/.... This will open a change spec for which you must provide a description:

Change: new

Client: warren

User:   admin

Status: new

Description:
        <enter description here>

Files:
        //support-tools/.git/COMMIT_EDITMSG     # add
        //support-tools/.git/FETCH_HEAD # add
        //support-tools/.git/HEAD       # add
        //support-tools/.git/ORIG_HEAD  # add
        //support-tools/.git/config     # add
        //support-tools/.git/description        # add

        # etc etc

You can see the files staged for adding with p4 opened.

Finally run p4 submit to load the files to the depo on the server.

To edit files

You must run p4 edit ${filename} before modifying files otherwise they are locked by default and set to read-only.

Configuring Sourcegraph to sync dogfood depots

Our documentation covers the requirements to sync to Sourcegraph, however for convienience it should be noted we have a user called buildkite on our dogfood instance whose session ticket will not expire. To generate the ticket for this account reference our shared 1Password.

To learn more about general p4 commands checkout this resource: https://www.perforce.com/perforce/doc.973/cmdguide/html/quicksta.htm

Helpful p4 flags

You can use -c in commands like p4 add to specify the client.

Local Perforce server

Joe has also developed an awesome Perforce server image that allows for local deployment of a Perforce server.

The following procedure runs the image:

  1. Start the server docker run --publish 1666:1666 sourcegraph/helix-p4d:2020.2

  2. Open a new terminal, configure the terminal session env:

export P4PORT=:1666
export P4USER=admin
  1. Ping the server from a separate terminal with p4 ping and enter the password below:
pass12349ers

(Note the value listed is from the dockerfile arg declaration)

GUI tools

You can download the Helix Admin Tool which makes it easier to configure users, permissions, etc.

Testing sub-repo permissions

Sub-repo permissions are still experimental and below are the steps required to start testing them locally against our dogfood perforce server.

Add this configuration to your external service configuration in dev private

"PERFORCE":
  [
    {
      "p4.port": "perforce.sgdev.org:1666",
      "p4.user": "admin",
      "p4.passwd": "REDACTED",
      "depots": ["//test-perms/"],
      "repositoryPathPattern": "perforce/{depot}",
      "authorization": { "subRepoPermissions": true },
    },
  ]

You can get the P4.passwd token above from running p4 -u admin login -p -a using the admin password from 1password. It expires every day so you’ll need to regenerate it occasionally.

Updated your local SG user to include the verified e-mail address of one of the users configured on the dogfood instance, for example alice@perforce.sgdev.org. Alice is a user in the our dogfood instance with permissions set against the //test-perms instance.

NOTE: Don’t modify the permissions in Perforce since they are used for integration testing.

You’ll also need Perforce and sub-repo permissions enabled in your site config under the experimentalFeatures section:

"experimentalFeatures": {
  "perforce": "enabled",
  "subRepoPermissions": { "enabled": true }
}

In order to not have to wait for a permissions sync to complete, you can force one with this mutation:

mutation {
  scheduleUserPermissionsSync(user: "YOUR_ID") {
    alwaysNil
  }
}

Your can find your user id with this query:

query {
  currentUser {
    id
  }
}

Once the permissions sync has completed you should see something like this in your local database:

sourcegraph=# select repo_id, user_id, path_includes, path_excludes from sub_repo_permissions;
 repo_id | user_id |   path_includes   |       path_excludes
---------+---------+-------------------+----------------------------
     238 |       1 | {**} | {Security/**}

Troubleshooting

git p4 is broken error

You may see this error when trying to run git p4:

fatal: 'p4' appears to be a git command, but we were not
able to execute it. Maybe git-p4 is broken?`

This can happen when the git p4 tool can’t locate the Python interpreter.

Here are the steps required to fix on MacOS assuming you use brew

  1. Locate git:
which git
/opt/homebrew/bin/git
  1. Check the symbolic link:
ls -l /opt/homebrew/bin/git
lrwxr-xr-x  1 username  admin  28 Aug 16  /opt/homebrew/bin/git -> ../Cellar/git/2.37.2/bin/git
  1. Change to the cellar directory and search for git-p4:
find . -name git-p4
./git/2.37.2/libexec/git-core/git-p4
  1. Edit git-p4 with your editor. Most likely the first line will be something like this:
#!/usr/bin/python
  1. Change it to this:
#!/usr/bin/python3
  1. git p4 should now work:
git p4
usage: /opt/homebrew/Cellar/git/2.37.2/libexec/git-core/git-p4 <command> [options]

valid commands: submit, commit, sync, rebase, clone, branches, unshelve

Try /opt/homebrew/Cellar/git/2.37.2/libexec/git-core/git-p4 <command> --help for command specific help.