Update spectral_scheduling.md

dmarx · May 11, 2024 · 46bacad · 46bacad
1 parent 16f3b90
commit 46bacad
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/spectral_scheduling.md b/spectral_scheduling.md
@@ -6,3 +6,5 @@ labels: experimental
 2. treat the eigenvalues as relative sampling weight, sample a basis dimension/component each batch
 3. sample a batch of data, using that datum's value in the sampled component as their relative likelihood (threshold this)
 4. incorporate a temperature term so we can sample disproportionately from the important components in early training, progressively increasing the weight of the tail probabilities.
+
+can we learn this online?