Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prep release #266

Merged
merged 3 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] - ReleaseDate
### Added
- [PR#261](https://github.com/EmbarkStudios/cargo-about/pull/261) resolved [#246](https://github.com/EmbarkStudios/cargo-about/issues/246) by adding an `--offline` (as well as `--locked` and `--frozen`) option to the `generate` command.
- [PR#266](https://github.com/EmbarkStudios/cargo-about/pull/266) resolved [#230](https://github.com/EmbarkStudios/cargo-about/issues/230) by adding a `--target` option to `generate`, allowing one to specify one or more targets to filter the dependency graph by, overriding the `targets` configuration option.

### Changed
- [PR#262](https://github.com/EmbarkStudios/cargo-about/pull/262) resolved [#258](https://github.com/EmbarkStudios/cargo-about/issues/258) by using LTO for release builds, slightly decreasing binary sizes.

### Fixed
- [PR#263](https://github.com/EmbarkStudios/cargo-about/pull/263) resolved [#238](https://github.com/EmbarkStudios/cargo-about/issues/238) by adding the `native-certs` feature to use the native certificate store. This feature is not enabled by default as it is only required for corporate environments that man in the middle network traffic.
- [PR#265](https://github.com/EmbarkStudios/cargo-about/pull/265) resolved [#198](https://github.com/EmbarkStudios/cargo-about/issues/198) by detecting if the parent process is powershell and exiting with an error if cargo-about's output is being redirected instead of using the `-o` option, as powershell is terrible and doesn't use utf-8 encoding by default.
- [PR#266](https://github.com/EmbarkStudios/cargo-about/pull/266) resolved [#222](https://github.com/EmbarkStudios/cargo-about/issues/222) by adding some additional documentation on <https://clearlydefined.io>.

## [0.6.4] - 2024-08-12
### Fixed
Expand Down
2 changes: 2 additions & 0 deletions docs/src/cli/clarify.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
<!-- markdownlint-disable no-duplicate-heading -->

# clarify

Computes a clarification for a file
Expand Down
2 changes: 1 addition & 1 deletion docs/src/cli/generate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Prevents Cargo and `cargo-about` from accessing the network for any reason. With

Beware that this may result in different dependency resolution than online mode. Cargo will restrict itself to crates that are downloaded locally, even if there might be a newer version as indicated in the local copy of the index. See the cargo-fetch(1) command to download dependencies before going offline.

`cargo-about` will also not query clearlydefined.io for license information, meaning that user provided clarifications won't be used, and some ambiguous/complicated license files might be missed by `cargo-about`. Additionally, clarifications that use license files from the crate's source repository will not be applied, meaning that `cargo-about` will fallback to using the default license text rather than the one in the source repository, losing eg. copyright or other unique information.
`cargo-about` will also not query <https://clearlydefined.io> for license information, meaning that user provided clarifications won't be used, and some ambiguous/complicated license files might be missed by `cargo-about`. Additionally, clarifications that use license files from the crate's source repository will not be applied, meaning that `cargo-about` will fallback to using the default license text rather than the one in the source repository, losing eg. copyright or other unique information.

### [`--frozen`](https://doc.rust-lang.org/cargo/commands/cargo-fetch.html#option-cargo-fetch---frozen)

Expand Down
16 changes: 14 additions & 2 deletions docs/src/cli/generate/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,21 @@ ignore-transitive-dependencies = true

## The `no-clearly-defined` field (optional)

If true, will not attempt to lookup licensing information for any crate from <https://clearlydefined.io>, only user clarifications, workarounds, and local file scanning will be used to determine licensing information.
If true, will not attempt to lookup licensing information for any crate from [clearlydefined.io], only user clarifications, workarounds, and local file scanning will be used to determine licensing information.

By default, `cargo-about` will use [clearlydefined.io] to augment the license information that can be gathered by scanning local files, as it has more advanced license detection (eg. it can detect multiple license in the same file unlike askalono), and can have [curations](https://docs.clearlydefined.io/docs/get-involved/data-curation) applied that benefit all users of a crate, rather than the project-specific clarifications supported by `cargo-about`.

[clearlydefined.io] does have some downsides however, in that it is an external source of information that can be missing or updated, which can result in different output given the same dependency graph input.

It will also show warnings for when the license information for a crate cannot be retrieved, the most common of which is

> the definition for <crate> has not been harvested

which indicates that the particular crate version has not been scanned and indexed by [clearlydefined.io] yet. Simply by making a request for a crate version from cargo-about, [clearlydefined.io] will automatically queue it to be harvested, but depending on load may take several hours or more before it is available.

## The `filter-noassertion` field (optional)

If using <https://clearlydefined.io> to gather license information, that service will conservatively add [`NOASSERTION`](https://docs.clearlydefined.io/curation-guidelines) to the expression for files that contain license like data, but an SPDX license ID could not be confidently ascribed to it. This can result in the license expression for the crate to contain 1 or more `NOASSERTION` identifiers, which would require the user to accept that (not really valid) ID to pass the license check. By setting this field to `true`, files that have a `NOASSERTION` id will instead be scanned locally, which will generally either figure out the license, or else skip that file.
If using [clearlydefined.io] to gather license information, that service will conservatively add [`NOASSERTION`](https://docs.clearlydefined.io/curation-guidelines) to the expression for files that contain license like data, but an SPDX license ID could not be confidently ascribed to it. This can result in the license expression for the crate to contain 1 or more `NOASSERTION` identifiers, which would require the user to accept that (not really valid) ID to pass the license check. By setting this field to `true`, files that have a `NOASSERTION` id will instead be scanned locally, which will generally either figure out the license, or else skip that file.

For a real world example of what this looks like, [`webpki:0.22.0`](https://crates.io/crates/webpki/0.22.0)'s [LICENSE](https://clearlydefined.io/file/5b698ca13897be3afdb7174256fa1574f8c6892b8bea1a66dd6469d3fe27885a) file is an ISC license, however it has a preamble that is not part of the ISC license that trips up clearly defined's inspection, causing it to be attributed with `ISC AND NOASSERTION`. Locally scanning the file will be more tolerant and just attribute it with `ISC`.

Expand Down Expand Up @@ -197,3 +207,5 @@ In some cases, crates concatenate multiple licenses together into a single file,
##### The `end` field (optional)

Just as with start, this is just a simple substring find, however, it will only match text that comes _after_ the position the start text (or beginning of the file) was found.

[clearlydefined.io]: https://clearlydefined.io
13 changes: 10 additions & 3 deletions src/cargo-about/generate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,12 @@ pub struct Args {
/// Do not activate the `default` feature
#[clap(long)]
no_default_features: bool,
/// The target triples to use for dependency graph filtering.
///
/// Overrides the `targets` configuration value, and note that unlike cargo
/// itself this can take multiple targets instead of just one.
#[clap(long)]
target: Vec<String>,
/// Run without accessing the network.
///
/// In addition to cargo not fetching crates, this will mean that only
Expand All @@ -63,13 +69,13 @@ pub struct Args {
/// copyright information in the license that would be retrieved from
/// the original git repo for the crate in question
#[arg(long)]
pub(crate) offline: bool,
offline: bool,
/// Assert that `Cargo.lock` will remain unchanged
#[arg(long)]
pub(crate) locked: bool,
locked: bool,
/// Equivalent to specifying both `--locked` and `--offline`
#[arg(long)]
pub(crate) frozen: bool,
frozen: bool,
/// The path of the Cargo.toml for the root crate.
///
/// Defaults to the current crate or workspace in the current working directory
Expand Down Expand Up @@ -194,6 +200,7 @@ pub fn cmd(args: Args, color: crate::Color) -> anyhow::Result<()> {
offline: args.offline,
},
&cfg,
&args.target,
));
});
s.spawn(|_| {
Expand Down
12 changes: 11 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ impl std::ops::Deref for Krate {

pub type Krates = krates::Krates<Krate>;

#[allow(clippy::too_many_arguments)]
pub fn get_all_crates(
cargo_toml: &krates::Utf8Path,
no_default_features: bool,
Expand All @@ -102,6 +103,7 @@ pub fn get_all_crates(
workspace: bool,
lock_opts: krates::LockOptions,
cfg: &licenses::config::Config,
target_overrdes: &[String],
) -> anyhow::Result<Krates> {
let mut mdc = krates::Cmd::new();
mdc.manifest_path(cargo_toml);
Expand Down Expand Up @@ -139,7 +141,15 @@ pub fn get_all_crates(
builder.ignore_kind(krates::DepKind::Build, krates::Scope::NonWorkspace);
}

builder.include_targets(cfg.targets.iter().map(|triple| (triple.as_str(), vec![])));
if target_overrdes.is_empty() {
builder.include_targets(cfg.targets.iter().map(|triple| (triple.as_str(), vec![])));
} else {
builder.include_targets(
target_overrdes
.iter()
.map(|triple| (triple.as_str(), vec![])),
);
}

let graph = builder.build(mdc, |filtered: cm::Package| {
if let Some(src) = filtered.source {
Expand Down