Rollup merge of rust-lang#41141 - michaelwoerister:direct-metadata-ich-final, r=nikomatsakis

ICH: Replace old, transitive metadata hashing with direct hashing approach.

This PR replaces the old crate metadata hashing strategy with a new one that directly (but stably) hashes all values we encode into the metadata. Previously we would track what data got accessed during metadata encoding and then hash the input nodes (HIR and upstream metadata) that were transitively reachable from the accessed data. While this strategy was sound, it had two major downsides:

1. It was susceptible to generating false positives, i.e. some input node might have changed without actually affecting the content of the metadata. That metadata entry would still show up as changed.
2. It was susceptible to quadratic blow-up when many metadata nodes shared the same input nodes, which would then get hashed over and over again.

The new method does not have these disadvantages and it's also a first step towards caching more intermediate results in the compiler.

Metadata hashing/cross-crate incremental compilation is still kept behind the `-Zincremental-cc` flag even after this PR. Once the new method has proven itself with more tests, we can remove the flag and enable cross-crate support by default again.

r? @nikomatsakis
cc @rust-lang/compiler

Author: TimNN

src/librustc/ich/hcx.rs

@@ -13,8 +13,10 @@ use hir::def_id::DefId;
 use ich::{self, CachingCodemapView};
 use session::config::DebugInfoLevel::NoDebugInfo;
 use ty;
+use util::nodemap::NodeMap;
 
 use std::hash as std_hash;
+use std::collections::{HashMap, HashSet};
 
 use syntax::ast;
 use syntax::attr;
@@ -296,3 +298,53 @@ impl<'a, 'tcx> HashStable<StableHashingContext<'a, 'tcx>> for Span {
         }
     }
 }
+
+pub fn hash_stable_hashmap<'a, 'tcx, K, V, R, SK, F, W>(hcx: &mut StableHashingContext<'a, 'tcx>,
+                                                        hasher: &mut StableHasher<W>,
+                                                        map: &HashMap<K, V, R>,
+                                                        extract_stable_key: F)
+    where K: Eq + std_hash::Hash,
+          V: HashStable<StableHashingContext<'a, 'tcx>>,
+          R: std_hash::BuildHasher,
+          SK: HashStable<StableHashingContext<'a, 'tcx>> + Ord + Clone,
+          F: Fn(&mut StableHashingContext<'a, 'tcx>, &K) -> SK,
+          W: StableHasherResult,
+{
+    let mut keys: Vec<_> = map.keys()
+                              .map(|k| (extract_stable_key(hcx, k), k))
+                              .collect();
+    keys.sort_unstable_by_key(|&(ref stable_key, _)| stable_key.clone());
+    keys.len().hash_stable(hcx, hasher);
+    for (stable_key, key) in keys {
+        stable_key.hash_stable(hcx, hasher);
+        map[key].hash_stable(hcx, hasher);
+    }
+}
+
+pub fn hash_stable_hashset<'a, 'tcx, K, R, SK, F, W>(hcx: &mut StableHashingContext<'a, 'tcx>,
+                                                     hasher: &mut StableHasher<W>,
+                                                     set: &HashSet<K, R>,
+                                                     extract_stable_key: F)
+    where K: Eq + std_hash::Hash,
+          R: std_hash::BuildHasher,
+          SK: HashStable<StableHashingContext<'a, 'tcx>> + Ord + Clone,
+          F: Fn(&mut StableHashingContext<'a, 'tcx>, &K) -> SK,
+          W: StableHasherResult,
+{
+    let mut keys: Vec<_> = set.iter()
+                              .map(|k| extract_stable_key(hcx, k))
+                              .collect();
+    keys.sort_unstable();
+    keys.hash_stable(hcx, hasher);
+}
+
+pub fn hash_stable_nodemap<'a, 'tcx, V, W>(hcx: &mut StableHashingContext<'a, 'tcx>,
+                                           hasher: &mut StableHasher<W>,
+                                           map: &NodeMap<V>)
+    where V: HashStable<StableHashingContext<'a, 'tcx>>,
+          W: StableHasherResult,
+{
+    hash_stable_hashmap(hcx, hasher, map, |hcx, node_id| {
+        hcx.tcx.hir.definitions().node_to_hir_id(*node_id).local_id
+    });
+}