# DSF

DSF: Sparsity-friendly Knowledge Distillation Through Feature Decoupling

## Introduction

This is the official implementation of the paper [Towards Optimal Sparse CNNs: Sparsity-friendly Knowledge Distillation Through Feature Decoupling].

## Requirements

- Python 
- PyTorch
- Torchvision

install dsp with pip:

```bash
pip install -e .
```

## Usage

### Training Teacher

```bash
python train_teacher.py --dataset cifar10 --model resnet110 --batch_size 256 --epochs 90
python train_teacher.py --dataset mini_imagenet --model resnet110 --batch_size 128 --epochs 90
```

### Training Student

```bash
python train_sparse_distill.py -r 0.5 -a 0.5 -b 0.3  --density 0.9  --epochs 240  --sparse_init ERK  --distill hint  --sparse
```