Create 0000_cargo_standard_directories.md

[spacekookie]

Initial draft of the RFC

[soc]

I think the most important contribution to this date is this comment by @brson here: rust-lang/cargo#5183 (comment).

If you take things step-by-step, I believe that the very first art of the RFC should suggest some cargo dirs sub-command, which introduces the notion that there might be different directories in the future, such that people can mentally (and code-wise) prepare for it.

[soc]

Because %LOCALAPPDATA% gets mentioned times and times again:

Unlike on Linux, Windows doesn't care about the value of %LOCALAPPDATA%, so pretending its value has any kind of meaning on that platform is wrong and broken.

SHGetKnownFolderPath is the sole source of truth.

(Libraries sorted this out already.)

[Screwtapello]

Technically, those aren't the default values of $XDG_CONFIG_HOME, $XDG_CACHE_HOME or $XDG_BIN_HOME. Perhaps it would be clearer to say something like:

Suggested change

	- `XDG_CONFIG_HOME = $HOME/.config/cargo`
	- `XDG_CACHE_HOME = $HOME/.cache/cargo`
	- `XDG_BIN_HOME = $HOME/.local/bin/cargo`
	- `CARGO_BIN_DIR` = `$XDG_BIN_HOME`(if set) or `$HOME/.local/bin`
	- `CARGO_CACHE_DIR` = `$XDG_CACHE_HOME/cargo` (if set) or `$HOME/.cache/cargo`
	- `CARGO_CONFIG_DIR` = `$XDG_CONFIG_HOME/cargo` (if set) or `$HOME/.config/cargo`

EDIT: I forgot suggestions were a thing, sorry!

[Screwtapello]

Actually, the most official-looking copy of the XDG Basedir spec I can find does not mention $XDG_BIN_HOME, it just hard-codes $HOME/.local/bin.

[eode]

@Screwtapello

Actually, the most official-looking copy of the XDG Basedir spec I can find does not mention $XDG_BIN_HOME, it just hard-codes $HOME/.local/bin.

I can definitively confirm that this is the current state of affairs with XDG -- there is no $XDG_BIN_HOME, they decided to go with just specifying the location in the XDG Basedir Spec alone. I argued against this throughout the whole discussion, but I didn't have a decisive use-case. That was before the following:

From Issue 1615

One issue that wasn't mentioned before is that $HOME might not be writable at all when an app (e.g. an IDE) is sandboxed in a container, for example flatpak. In these cases, cargo/rustup fails by default. Tinkering with environment variables is needed to make it work. Using XDG variables would make cargo/rustup work without additional configuration.

I ran into this issue when trying to setup Eclipse Corrosion in sandboxed Eclipse (eclipse-corrosion/corrosion#366).

If I'm correct, that's a definitive need for the 1$XDG_BIN_HOME variable. Can anyone else argue otherwise? ..because the XDG people will, and I'd like to be able to shoot those arguments down.

[soc]

I'm not sure I understand that – XDG's dir for executables is specified to be a sibling subdir of XDG_DATA_HOME, so if some things aren't writable, the binary dir not being writable will be the least of your concerns.

[Screwtapello]

If I'm correct, that's a definitive need for the $XDG_BIN_HOME variable. Can anyone else argue otherwise? ..because the XDG people will, and I'd like to be able to shoot those arguments down.

One alternative to an $XDG_BIN_HOME variable might be "the first writable entry on $PATH". Then application sandboxes like Flatpak could stick a sandboxed directory at the front of $PATH and be done.

A counter-counter-argument to that counter-argument might be that the same duality already exists between, say $XDG_CONFIG_HOME and $XDG_CONFIG_DIRS:

Config storage	Binary storage
$XDG_CONFIG_DIRS	$PATH
$XDG_CONFIG_HOME	???

[Jesse-Bakker]

In the context of this RFC, let's just follow the spec as is and move the discussion about the spec itself to the XDG issue tracker or elsewhere.

For the purposes of cargo, the bin dir would be separately overridable using $CARGO_BIN_DIR anyway

[soc]

Alternatively, you could use a library that deals with this.

[0x5c]

Eh ... sorry ... "subdir" → "derived". The rest of my comment still applies the same though.

@soc The spec hardcodes the bin dir as $HOME/.local/bin not $XDG_DATA_HOME/bin (which would be ~/.local/share/bin).
Even if the bin dir was somehow "derived" from the value of $XDG_DATA_HOME, it could lead to unexpected situations where a write is attempted in the parent dir of $XDG_DATA_HOME
Ie: Attempting to create bin/ in /unwritabledir/ if $XDG_DATA_HOME points to /unwritabledir/userdata/.

[soc]

@0x5c yeah, saw it, not sure if that was intentional or not. Not sure this is relevant for this PR anymore, as the directory haggling predictably derailed any other work on this PR.

Interestingly they also finally managed to standardize XDG_STATE_HOME after like 10 years ... I will update my libraries to support that.

[Screwtapello]

Suggested change

	By default these paths are as follows.
	If the `CARGO_*_DIR` environment variables are not set, Cargo will choose defaults for them based on on the XDG Basedir specification:

[soc]

That sounds like a good choice, simply linking to the spec/API on all three platforms.

[eode]

That also sounds good to me. We can update to include the $XDG_BIN_DIR when or if the XDG spec changes to include it.

[hcsch]

Is there any chance of including a section on the target directory currently present in workspaces / package directories after working with cargo build, cargo test or similar? In particular considering moving this directory and the files within it (with some unique name, perhaps derived from the path to the projects files) to the user local cache directory currently known to cargo (whatever that might end up being).
This would mostly concern its default location. I imagine it would still be valuable to leave this configurable with command line flags or a separate environment variable.

As others have mention (rust-lang/cargo#1734 (comment), rust-lang/cargo#1734 (comment)), keeping a directory of mostly temporary generated files alongside the source files at arbitrary locations in the users filesystem can also be a source of pain.

My personal use-case for this is using BTRFS snapshots to make efficient and fast backups of my home directory. While this method is very efficient it does not allow for arbitrary filtering rules to exclude directories such as target in a cargo workspace due to the technical details of filesystem snapshotting.
A solution to this is marking special fixed paths as separate from the main home filesystem (making it another subvolume in BTRFS's case). This however is only possible if most if not all temporary files (that are uninteresting / hindering for backups) are stored under a common ancestor path (e.g. ~/.cache).
While my use-case deals with BTRFS in particular, I've noticed other backup approaches often have similar issues due to limited filtering options (e.g. rsync can use .gitignore where available, but this fails for workspaces with git versioned crates, but itself no git versioning or .gitignore), and the inability to rule out target as a name for directories that should be backed up in general (somebody might want to use that name for other reasons).

This issue is not special to Rust and cargo, but rather to all build systems that store generated files alongside source files (or even worse interspersed with source files). I feel like Rust could have another benefit over other programming language ecosystems by getting this right.

A possible counter point worth mentioning would be the wide-reaching use of git versioning in the Rust ecosystem paired with git repository servers, making local backups somewhat unnecessary or perhaps better accomplished otherwise.
In my case I could probably store versioned code projects in a separate subvolume only for such projects that will not be included in backups.

If you feel that his might be out of scope for this RFC, do tell (and possibly mark this off-topic OSLT). I can also imagine this fitting in a standalone follow up RFC after this one. I however feel that this is an important point that might be worth bundling together with this proposal.

It might be worth pulling in opinions from people involved with rust-lang/cargo#6100.

[mathstuf]

For this specifically, target can be a symlink off into some ignored place. I've been doing this for years (and contributing fixes to projects that don't ignore it when it is a symlink. Anyone using cargo init after Jan 2018 has this patch to support it, but some are still around. And other gitignore templates don't have the change yet.