πŸ“Š Pertemuan 10 | Week 10

Evaluasi Model Unsupervised Learning

Metode validasi dan evaluasi kualitas clustering: internal metrics, external metrics, stability analysis, dan best practices untuk model unsupervised learning

⏱️ 3 Γ— 50 menit
🎯 Silhouette, Davies-Bouldin, Calinski-Harabasz
πŸ“ˆ Cluster Validation & Optimization

πŸ“½οΈ Slide Presentation

πŸ’‘ Cara Menggunakan Slide

Gunakan tombol panah atau klik untuk navigasi slide. Tekan tombol fullscreen untuk tampilan layar penuh. Slide dapat diakses kapan saja untuk review materi.

🎯 Tujuan Pembelajaran

Setelah menyelesaikan pertemuan ini, mahasiswa diharapkan mampu:

  1. Memahami tantangan evaluasi model unsupervised learning (tidak ada ground truth labels)
  2. Mengimplementasikan internal validation metrics (Silhouette Score, Davies-Bouldin Index, Calinski-Harabasz)
  3. Menerapkan external validation metrics ketika ground truth tersedia (Rand Index, Adjusted Rand Index, NMI)
  4. Menggunakan Elbow Method dan Gap Statistic untuk menentukan jumlah cluster optimal
  5. Melakukan stability analysis dan cross-validation untuk clustering
  6. Menginterpretasi dan membandingkan hasil dari berbagai metrics
  7. Memilih metrics yang appropriate untuk konteks geofisika spesifik
  8. Mengidentifikasi dan menghindari common pitfalls dalam evaluasi clustering

πŸ“Š Internal Validation Metrics

Internal metrics mengevaluasi kualitas clustering berdasarkan data itu sendiri tanpa memerlukan ground truth labels. Metrics ini mengukur seberapa baik clusters terbentuk berdasarkan compactness (within-cluster similarity) dan separation (between-cluster dissimilarity).

🎯

Silhouette Score

s(i) = (b(i) - a(i)) / max(a(i), b(i))

Mengukur seberapa mirip suatu data point dengan cluster-nya sendiri dibanding cluster lain. a(i) = avg distance dalam cluster, b(i) = avg distance ke cluster terdekat.

Range: -1 to +1 | Optimal: mendekati +1
πŸ“

Davies-Bouldin Index

DB = (1/k) Ξ£ max((Οƒα΅’ + Οƒβ±Ό) / d(cα΅’,cβ±Ό))

Rasio rata-rata similarity antara setiap cluster dengan cluster yang paling mirip dengannya. Mengukur average worst-case similarity.

Range: 0 to ∞ | Optimal: mendekati 0
πŸ“ˆ

Calinski-Harabasz Index

CH = (SSB/(k-1)) / (SSW/(n-k))

Variance ratio criterion: rasio between-cluster dispersion (SSB) terhadap within-cluster dispersion (SSW). Higher is better.

Range: 0 to ∞ | Optimal: nilai tinggi
🎲

Inertia (WCSS)

Inertia = Ξ£ Ξ£ ||xα΅’ - ΞΌβ‚–||Β²

Within-Cluster Sum of Squares. Total squared distance dari setiap point ke centroid cluster-nya. Digunakan dalam Elbow Method untuk K-Means.

Range: 0 to ∞ | Optimal: nilai rendah
πŸ”

Dunn Index

DI = min(Ξ΄(Cα΅’,Cβ±Ό)) / max(Ξ”β‚–)

Rasio minimum inter-cluster distance terhadap maximum intra-cluster distance. Mengukur compact and well-separated clusters.

Range: 0 to ∞ | Optimal: nilai tinggi
πŸ“Š

Gap Statistic

Gap(k) = E[log(Wβ‚–)] - log(Wβ‚–)

Membandingkan within-cluster dispersion dengan expected value dari null reference distribution. Optimal K = smallest k dimana Gap(k) β‰₯ Gap(k+1) - s_{k+1}.

Optimal: K dengan gap terbesar
⚠️ Important Notes
  • Tidak ada single best metric: Berbeda metrics bisa memberikan hasil berbeda untuk data yang sama
  • Use multiple metrics: Kombinasikan beberapa metrics untuk validasi yang robust
  • Domain knowledge: Interpretasi harus dikombinasikan dengan pemahaman geologis/geofisika
  • Computational cost: Beberapa metrics (Gap Statistic, Dunn) expensive untuk large datasets

πŸ”— External Validation Metrics

External metrics digunakan ketika ground truth labels tersedia (supervised evaluation of unsupervised learning). Berguna untuk:

  • Validasi clustering terhadap known facies/lithology labels
  • Benchmarking berbagai algoritma clustering
  • Semi-supervised scenarios dalam geofisika
🎲

Rand Index (RI)

RI = (TP + TN) / (TP + FP + FN + TN)

Mengukur similarity antara dua clustering (predicted vs true). TP = pairs correctly in same cluster, TN = pairs correctly in different clusters.

Range: 0 to 1 | Optimal: 1
✨

Adjusted Rand Index (ARI)

ARI = (RI - E[RI]) / (max(RI) - E[RI])

Adjusted untuk chance: mengkoreksi Rand Index terhadap random clustering. ARI = 0 untuk random clustering, ARI = 1 untuk perfect match.

Range: -1 to 1 | Optimal: 1
πŸ”’

Normalized Mutual Information

NMI = 2 * MI(U,V) / (H(U) + H(V))

Mengukur mutual information antara cluster assignments, normalized by entropy. Information-theoretic measure of clustering similarity.

Range: 0 to 1 | Optimal: 1
πŸ“

Fowlkes-Mallows Index

FMI = √(TP/(TP+FP) * TP/(TP+FN))

Geometric mean of precision and recall for pairwise comparisons. Balanced measure antara cluster purity dan completeness.

Range: 0 to 1 | Optimal: 1
πŸ”¬ Aplikasi dalam Geofisika

External metrics sangat berguna untuk:

  • Validasi clustering well log terhadap core description atau petrophysical cutoffs
  • Evaluasi seismic facies clustering dengan interpreted horizons atau well ties
  • Benchmarking unsupervised methods terhadap supervised baselines
  • Semi-supervised scenarios: partial labels dari few wells untuk validate clustering di area lain

🎯 Menentukan Jumlah Cluster Optimal

1. Elbow Method

Plot inertia (WCSS) vs number of clusters. Optimal K berada di "elbow" - titik dimana penambahan cluster tidak signifikan menurunkan inertia.

Pros: Simple, intuitive, computationally cheap

Cons: Subjective (elbow tidak selalu jelas), hanya untuk K-Means, tidak ada statistical test

2. Silhouette Analysis

Plot silhouette score untuk berbagai nilai K. Pilih K dengan silhouette score tertinggi. Silhouette plot per-cluster juga useful untuk identify poorly-separated clusters.

Pros: Quantitative, visual inspection of individual clusters, works for any distance-based algorithm

Cons: Computational cost O(nΒ²), assumes convex clusters

3. Gap Statistic

Membandingkan total within-cluster variation dengan expected value dari null reference distribution (uniform random data).

Pros: Statistical justification, formal test, tidak bergantung pada visual inspection

Cons: Computationally expensive (requires bootstrapping), sensitive to reference distribution choice

4. Domain Knowledge & Geological Context

βœ… Best Practice untuk Geofisika

Kombinasikan statistical metrics dengan geological understanding:

  • Well log clustering: Expected number of facies dari stratigraphic interpretation
  • Seismic facies: Number of seismic units dari sequence stratigraphy
  • Reservoir zonation: Flow units dari core analysis atau RFT data
  • Balance: Terlalu sedikit clusters β†’ under-segmentation, terlalu banyak β†’ noise/overfitting

βš–οΈ Perbandingan Metrics

Metric Type Best Value Computational Cost Use Case
Silhouette Score Internal +1 (higher better) O(nΒ²) General purpose, any distance-based clustering
Davies-Bouldin Internal 0 (lower better) O(nΒ·k) Fast, good for K-Means, convex clusters
Calinski-Harabasz Internal ∞ (higher better) O(n·k) Fast, works well with K-Means
Inertia (WCSS) Internal 0 (lower better) O(nΒ·k) K-Means only, use with Elbow Method
Gap Statistic Internal Peak value O(BΒ·nΒ·k) Finding optimal K, statistical justification
Adjusted Rand Index External 1 (higher better) O(n) When ground truth available, benchmarking
NMI External 1 (higher better) O(nΒ·k) Information-theoretic, balanced measure

✨ Best Practices & Common Pitfalls

βœ… Recommended Workflow

  1. Exploratory Analysis: Visualize data dengan PCA/t-SNE, understand inherent structure
  2. Determine K Range: Use domain knowledge untuk set realistic K range (e.g., K=2 to 10)
  3. Multiple Metrics: Compute Silhouette, Davies-Bouldin, Calinski-Harabasz untuk K range
  4. Visual Inspection: Plot metrics, look for consensus (berbagai metrics agree pada K tertentu)
  5. Stability Analysis: Run clustering multiple times dengan different initializations, check consistency
  6. Geological Validation: Interpret clusters dalam konteks geologi, verify dengan domain expert
  7. Cross-validation: Jika ada ground truth partial, use external metrics untuk validate

⚠️ Common Pitfalls to Avoid

Kesalahan Umum dalam Evaluasi Clustering
  • Over-reliance on single metric: Berbeda metrics bisa contradict - use multiple!
  • Ignoring data characteristics: K-Means metrics assume spherical clusters - inappropriate untuk DBSCAN results
  • Not scaling features: Metrics sensitive to feature scales - always standardize!
  • Optimizing K to extreme: Very high K β†’ overfitting to noise, very low K β†’ under-segmentation
  • Forgetting domain context: Metrics hanya tools - geological interpretation is key!
  • Not checking stability: Single run bisa unstable - verify dengan multiple random seeds
  • Comparing different algorithms unfairly: Use same metrics, same preprocessing, same data splits

πŸ’» Implementation Example

Contoh Python code untuk evaluate clustering menggunakan multiple metrics:

🐍 Code Snippet
from sklearn.metrics import silhouette_score, davies_bouldin_score, calinski_harabasz_score
from sklearn.cluster import KMeans
import numpy as np

# Evaluate clustering untuk range K
results = []
K_range = range(2, 11)

for k in K_range:
    kmeans = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = kmeans.fit_predict(X_scaled)
    
    # Compute metrics
    sil = silhouette_score(X_scaled, labels)
    db = davies_bouldin_score(X_scaled, labels)
    ch = calinski_harabasz_score(X_scaled, labels)
    inertia = kmeans.inertia_
    
    results.append({
        'K': k,
        'Silhouette': sil,
        'Davies-Bouldin': db,
        'Calinski-Harabasz': ch,
        'Inertia': inertia
    })
    
    print(f"K={k}: Sil={sil:.3f}, DB={db:.3f}, CH={ch:.1f}")

# Find optimal K based on consensus
# Higher Silhouette, Lower DB, Higher CH

Expected Output:

K=2: Sil=0.587, DB=0.842, CH=1024.3
K=3: Sil=0.631, DB=0.735, CH=1189.7
K=4: Sil=0.624, DB=0.698, CH=1256.4 ← Optimal!
K=5: Sil=0.603, DB=0.742, CH=1198.2
...

πŸ“– Referensi & Resources

Papers & Books:

  • Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation of cluster analysis
  • Davies, D. L., & Bouldin, D. W. (1979). A Cluster Separation Measure
  • CaliΕ„ski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis
  • Hubert, L., & Arabie, P. (1985). Comparing partitions (ARI paper)

Documentation:

Practice:

πŸ“ Evaluasi

Tugas:

  1. Implement Praktikum 03 dengan menambahkan ALL validation metrics (Silhouette, DB, CH, Gap Statistic)
  2. Plot metrics vs K dan tentukan optimal K berdasarkan consensus
  3. Compare K-Means, Hierarchical, dan DBSCAN pada same dataset dengan same metrics
  4. Interpret hasil dalam konteks geological facies
  5. Write report: methodology, results, interpretation, limitations

Penilaian:

  • Implementation correctness (30%)
  • Proper use of multiple metrics (25%)
  • Analysis & interpretation (25%)
  • Geological validation (20%)