android_build/tools/compliance/policy_policy.go

290 lines
11 KiB
Go
Raw Normal View History

// Copyright 2021 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package compliance
import (
"regexp"
"strings"
)
var (
// RecognizedAnnotations identifies the set of annotations that have
// meaning for compliance policy.
RecognizedAnnotations = map[string]string{
// used in readgraph.go to avoid creating 1000's of copies of the below 3 strings.
"static": "static",
"dynamic": "dynamic",
"toolchain": "toolchain",
}
// SafePathPrefixes maps the path prefixes presumed not to contain any
// proprietary or confidential pathnames to whether to strip the prefix
// from the path when used as the library name for notices.
SafePathPrefixes = map[string]bool{
"external/": true,
"art/": false,
"build/": false,
"cts/": false,
"dalvik/": false,
"developers/": false,
"development/": false,
"frameworks/": false,
"packages/": true,
"prebuilts/": false,
"sdk/": false,
"system/": false,
"test/": false,
"toolchain/": false,
"tools/": false,
}
// SafePrebuiltPrefixes maps the regular expression to match a prebuilt
// containing the path of a safe prefix to the safe prefix.
SafePrebuiltPrefixes = make(map[*regexp.Regexp]string)
// ImpliesUnencumbered lists the condition names representing an author attempt to disclaim copyright.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesUnencumbered = LicenseConditionSet(UnencumberedCondition)
// ImpliesPermissive lists the condition names representing copyrighted but "licensed without policy requirements".
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesPermissive = LicenseConditionSet(PermissiveCondition)
// ImpliesNotice lists the condition names implying a notice or attribution policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesNotice = LicenseConditionSet(UnencumberedCondition | PermissiveCondition | NoticeCondition | ReciprocalCondition |
RestrictedCondition | RestrictedClasspathExceptionCondition | WeaklyRestrictedCondition |
ProprietaryCondition | ByExceptionOnlyCondition)
// ImpliesReciprocal lists the condition names implying a local source-sharing policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesReciprocal = LicenseConditionSet(ReciprocalCondition)
// Restricted lists the condition names implying an infectious source-sharing policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesRestricted = LicenseConditionSet(RestrictedCondition | RestrictedClasspathExceptionCondition | WeaklyRestrictedCondition)
// ImpliesProprietary lists the condition names implying a confidentiality policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesProprietary = LicenseConditionSet(ProprietaryCondition)
// ImpliesByExceptionOnly lists the condition names implying a policy for "license review and approval before use".
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesByExceptionOnly = LicenseConditionSet(ProprietaryCondition | ByExceptionOnlyCondition)
// ImpliesPrivate lists the condition names implying a source-code privacy policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesPrivate = LicenseConditionSet(ProprietaryCondition)
// ImpliesShared lists the condition names implying a source-code sharing policy.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
ImpliesShared = LicenseConditionSet(ReciprocalCondition | RestrictedCondition | RestrictedClasspathExceptionCondition | WeaklyRestrictedCondition)
)
var (
anyLgpl = regexp.MustCompile(`^SPDX-license-identifier-LGPL.*`)
versionedGpl = regexp.MustCompile(`^SPDX-license-identifier-GPL-\p{N}.*`)
genericGpl = regexp.MustCompile(`^SPDX-license-identifier-GPL$`)
ccBySa = regexp.MustCompile(`^SPDX-license-identifier-CC-BY.*-SA.*`)
)
func init() {
for prefix := range SafePathPrefixes {
if prefix == "prebuilts/" {
continue
}
r := regexp.MustCompile("^prebuilts/[^ ]*/" + prefix)
SafePrebuiltPrefixes[r] = prefix
}
}
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// LicenseConditionSetFromNames returns a set containing the recognized `names` and
// silently ignoring or discarding the unrecognized `names`.
func LicenseConditionSetFromNames(tn *TargetNode, names ...string) LicenseConditionSet {
cs := NewLicenseConditionSet()
for _, name := range names {
if name == "restricted" {
if 0 == len(tn.LicenseKinds()) {
cs = cs.Plus(RestrictedCondition)
continue
}
hasLgpl := false
hasClasspath := false
hasGeneric := false
for _, kind := range tn.LicenseKinds() {
if strings.HasSuffix(kind, "-with-classpath-exception") {
cs = cs.Plus(RestrictedClasspathExceptionCondition)
hasClasspath = true
} else if anyLgpl.MatchString(kind) {
cs = cs.Plus(WeaklyRestrictedCondition)
hasLgpl = true
} else if versionedGpl.MatchString(kind) {
cs = cs.Plus(RestrictedCondition)
} else if genericGpl.MatchString(kind) {
hasGeneric = true
} else if kind == "legacy_restricted" || ccBySa.MatchString(kind) {
cs = cs.Plus(RestrictedCondition)
} else {
cs = cs.Plus(RestrictedCondition)
}
}
if hasGeneric && !hasLgpl && !hasClasspath {
cs = cs.Plus(RestrictedCondition)
}
continue
}
if lc, ok := RecognizedConditionNames[name]; ok {
cs |= LicenseConditionSet(lc)
}
}
return cs
}
// Resolution happens in three phases:
//
// 1. A bottom-up traversal propagates (restricted) license conditions up to
// targets from dendencies as needed.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// 2. For each condition of interest, a top-down traversal propagates
// (restricted) conditions down from targets into linked dependencies.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// 3. Finally, a walk of the shipped target nodes attaches resolutions to the
// ancestor nodes from the root down to and including the first non-container.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// e.g. If a disk image contains a binary bin1 that links a library liba, the
// notice requirement for liba gets attached to the disk image and to bin1.
// Because liba doesn't actually get shipped as a separate artifact, but only
// as bits in bin1, it has no actions 'attached' to it. The actions attached
// to the image and to bin1 'act on' liba by providing notice.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// The behavior of the 3 phases gets controlled by the 3 functions below.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// The first function controls what happens during the bottom-up propagation.
// Restricted conditions propagate up all non-toolchain dependencies; except,
// some do not propagate up dynamic links, which may depend on whether the
// modules are independent.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// The second function controls what happens during the top-down propagation.
// Restricted conditions propagate down as above with the added caveat that
// inherited restricted conditions do not propagate from pure aggregates to
// their dependencies.
//
// The final function controls which conditions apply/get attached to ancestors
// depending on the types of dependencies involved. All conditions apply across
// normal derivation dependencies. No conditions apply across toolchain
// dependencies. Some restricted conditions apply across dynamic link
// dependencies.
//
// Not all restricted licenses are create equal. Some have special rules or
// exceptions. e.g. LGPL or "with classpath excption".
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// depConditionsPropagatingToTarget returns the conditions which propagate up an
// edge from dependency to target.
//
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// This function sets the policy for the bottom-up propagation and how conditions
// flow up the graph from dependencies to targets.
//
// If a pure aggregation is built into a derivative work that is not a pure
// aggregation, per policy it ceases to be a pure aggregation in the context of
// that derivative work. The `treatAsAggregate` parameter will be false for
// non-aggregates and for aggregates in non-aggregate contexts.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
func depConditionsPropagatingToTarget(lg *LicenseGraph, e *TargetEdge, depConditions LicenseConditionSet, treatAsAggregate bool) LicenseConditionSet {
result := LicenseConditionSet(0x0000)
if edgeIsDerivation(e) {
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result |= depConditions & ImpliesRestricted
return result
}
if !edgeIsDynamicLink(e) {
return result
}
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result |= depConditions & LicenseConditionSet(RestrictedCondition)
if 0 != (depConditions&LicenseConditionSet(RestrictedClasspathExceptionCondition)) && !edgeNodesAreIndependentModules(e) {
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result |= LicenseConditionSet(RestrictedClasspathExceptionCondition)
}
return result
}
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// targetConditionsPropagatingToDep returns the conditions which propagate down
// an edge from target to dependency.
//
// This function sets the policy for the top-down traversal and how conditions
// flow down the graph from targets to dependencies.
//
// If a pure aggregation is built into a derivative work that is not a pure
// aggregation, per policy it ceases to be a pure aggregation in the context of
// that derivative work. The `treatAsAggregate` parameter will be false for
// non-aggregates and for aggregates in non-aggregate contexts.
func targetConditionsPropagatingToDep(lg *LicenseGraph, e *TargetEdge, targetConditions LicenseConditionSet, treatAsAggregate bool, conditionsFn TraceConditions) LicenseConditionSet {
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result := targetConditions
// reverse direction -- none of these apply to things depended-on, only to targets depending-on.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result = result.Minus(UnencumberedCondition, PermissiveCondition, NoticeCondition, ReciprocalCondition, ProprietaryCondition, ByExceptionOnlyCondition)
if !edgeIsDerivation(e) && !edgeIsDynamicLink(e) {
// target is not a derivative work of dependency and is not linked to dependency
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result = result.Difference(ImpliesRestricted)
return result
}
if treatAsAggregate {
// If the author of a pure aggregate licenses it restricted, apply restricted to immediate dependencies.
// Otherwise, restricted does not propagate back down to dependencies.
if !conditionsFn(e.target).MatchesAnySet(ImpliesRestricted) {
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result = result.Difference(ImpliesRestricted)
}
return result
}
if edgeIsDerivation(e) {
return result
}
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result = result.Minus(WeaklyRestrictedCondition)
if edgeNodesAreIndependentModules(e) {
result = result.Minus(RestrictedClasspathExceptionCondition)
}
return result
}
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
// conditionsAttachingAcrossEdge returns the subset of conditions in `universe`
// that apply across edge `e`.
//
// This function sets the policy for attaching actions to ancestor nodes in the
// final resolution walk.
func conditionsAttachingAcrossEdge(lg *LicenseGraph, e *TargetEdge, universe LicenseConditionSet) LicenseConditionSet {
result := universe
if edgeIsDerivation(e) {
return result
}
if !edgeIsDynamicLink(e) {
return NewLicenseConditionSet()
}
result &= LicenseConditionSet(RestrictedCondition | RestrictedClasspathExceptionCondition)
if 0 != (result&LicenseConditionSet(RestrictedClasspathExceptionCondition)) && edgeNodesAreIndependentModules(e) {
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
result &= LicenseConditionSet(RestrictedCondition)
}
return result
}
// edgeIsDynamicLink returns true for edges representing shared libraries
// linked dynamically at runtime.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
func edgeIsDynamicLink(e *TargetEdge) bool {
return e.annotations.HasAnnotation("dynamic")
}
// edgeIsDerivation returns true for edges where the target is a derivative
// work of dependency.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
func edgeIsDerivation(e *TargetEdge) bool {
isDynamic := e.annotations.HasAnnotation("dynamic")
isToolchain := e.annotations.HasAnnotation("toolchain")
return !isDynamic && !isToolchain
}
// edgeNodesAreIndependentModules returns true for edges where the target and
// dependency are independent modules.
Performance and scale. Defer edge creation. Don't create edges until the count is known to avoid repeated allocate+ copy operatios. Limit resolutions. Allow only a single resolution condition set per target, and overwrite intermediate results. Reduces memory and obviates allocations. Propagate fewer conditions. Instead of propagating notice conditions to parents in graph during initial resolve, leave them on leaf node, and attach to ancestors in the final walk. Reduces copies. Parallelize resolutions. Use goroutines, mutexes, and waitgroups to resolve branches of the graph in parallel. Makes better use of available cores. Don't accumulate resolutions inside non-containers. During the final resolution walk, only attach actions to ancestors from the root down until the 1st non-aggregate. Prevents an explosion of copies in the lower levels of the graph. Drop origin for scale. Tracking the origin of every potential origin for every restricted condition does not scale. By dropping origin, propagating from top to bottom can prune many redundant paths avoiding an exponential explosion. Conditions as bitmask. Use bit masks for license conditions and condition sets. Reduces maps and allocations. Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Test: m systemlicense Test: m listshare; out/soong/host/linux-x86/bin/listshare ... Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ... Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ... Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ... where ... is the path to the .meta_lic file for the system image. In my case if $ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD) ... can be expressed as: ${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-10 21:50:57 +00:00
func edgeNodesAreIndependentModules(e *TargetEdge) bool {
return e.target.PackageName() != e.dependency.PackageName()
}