research_publications.rst

Research and Publications
=========================

We are intensively working on both tool chain and research to make automatic model design and tuning really practical and powerful. On the one hand, our main work is tool chain oriented development. On the other hand, our research works aim to improve this tool chain, rethink challenging problems in AutoML (on both system and algorithm) and propose elegant solutions. Below we list some of our research works, we encourage more research works on this topic and encourage collaboration with us.

System Research
---------------


* `SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute <https://www.usenix.org/system/files/osdi22-zheng-ningxin.pdf>`__

.. code-block:: bibtex

   @inproceedings{zheng2022sparta,
     title={$\{$SparTA$\}$:$\{$Deep-Learning$\}$ Model Sparsity via $\{$Tensor-with-Sparsity-Attribute$\}$},
     author={Zheng, Ningxin and Lin, Bin and Zhang, Quanlu and Ma, Lingxiao and Yang, Yuqing and Yang, Fan and Wang, Yang and Yang, Mao and Zhou, Lidong},
     booktitle={16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22)},
     pages={213--232},
     year={2022}
   }


* `Retiarii: A Deep Learning Exploratory-Training Framework <https://www.usenix.org/system/files/osdi20-zhang_quanlu.pdf>`__

.. code-block:: bibtex

   @inproceedings{zhang2020retiarii,
     title={Retiarii: A Deep Learning Exploratory-Training Framework},
     author={Zhang, Quanlu and Han, Zhenhua and Yang, Fan and Zhang, Yuge and Liu, Zhe and Yang, Mao and Zhou, Lidong},
     booktitle={14th $\{$USENIX$\}$ Symposium on Operating Systems Design and Implementation ($\{$OSDI$\}$ 20)},
     pages={919--936},
     year={2020}
   }


* `AutoSys: The Design and Operation of Learning-Augmented Systems <https://www.usenix.org/system/files/atc20-liang-chieh-jan.pdf>`__

.. code-block:: bibtex

   @inproceedings{liang2020autosys,
     title={AutoSys: The Design and Operation of Learning-Augmented Systems},
     author={Liang, Chieh-Jan Mike and Xue, Hui and Yang, Mao and Zhou, Lidong and Zhu, Lifei and Li, Zhao Lucis and Wang, Zibo and Chen, Qi and Zhang, Quanlu and Liu, Chuanjie and others},
     booktitle={2020 $\{$USENIX$\}$ Annual Technical Conference ($\{$USENIX$\}$$\{$ATC$\}$ 20)},
     pages={323--336},
     year={2020}
   }


* `Gandiva: Introspective Cluster Scheduling for Deep Learning <https://www.usenix.org/system/files/osdi18-xiao.pdf>`__

.. code-block:: bibtex

   @inproceedings{xiao2018gandiva,
     title={Gandiva: Introspective cluster scheduling for deep learning},
     author={Xiao, Wencong and Bhardwaj, Romil and Ramjee, Ramachandran and Sivathanu, Muthian and Kwatra, Nipun and Han, Zhenhua and Patel, Pratyush and Peng, Xuan and Zhao, Hanyu and Zhang, Quanlu and others},
     booktitle={13th $\{$USENIX$\}$ Symposium on Operating Systems Design and Implementation ($\{$OSDI$\}$ 18)},
     pages={595--610},
     year={2018}
   }

Algorithm Research
------------------

New Algorithms
^^^^^^^^^^^^^^


* `Privacy-preserving Online AutoML for Domain-Specific Face Detection <https://openaccess.thecvf.com/content/CVPR2022/papers/Yan_Privacy-Preserving_Online_AutoML_for_Domain-Specific_Face_Detection_CVPR_2022_paper.pdf>`__

.. code-block:: bibtex

   @inproceedings{yan2022privacy,
     title={Privacy-preserving Online AutoML for Domain-Specific Face Detection},
     author={Yan, Chenqian and Zhang, Yuge and Zhang, Quanlu and Yang, Yaming and Jiang, Xinyang and Yang, Yuqing and Wang, Baoyuan},
     booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
     pages={4134--4144},
     year={2022}
   }


* `TextNAS: A Neural Architecture Search Space Tailored for Text Representation <https://arxiv.org/pdf/1912.10729.pdf>`__

.. code-block:: bibtex

   @inproceedings{wang2020textnas,
     title={TextNAS: A Neural Architecture Search Space Tailored for Text Representation.},
     author={Wang, Yujing and Yang, Yaming and Chen, Yiren and Bai, Jing and Zhang, Ce and Su, Guinan and Kou, Xiaoyu and Tong, Yunhai and Yang, Mao and Zhou, Lidong},
     booktitle={AAAI},
     pages={9242--9249},
     year={2020}
   }


* `Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search <https://papers.nips.cc/paper/2020/file/d072677d210ac4c03ba046120f0802ec-Paper.pdf>`__

.. code-block:: bibtex

   @article{peng2020cream,
     title={Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search},
     author={Peng, Houwen and Du, Hao and Yu, Hongyuan and Li, Qi and Liao, Jing and Fu, Jianlong},
     journal={Advances in Neural Information Processing Systems},
     volume={33},
     year={2020}
   }


* `Metis: Robustly tuning tail latencies of cloud systems <https://www.usenix.org/system/files/conference/atc18/atc18-li-zhao.pdf>`__

.. code-block:: bibtex

   @inproceedings{li2018metis,
     title={Metis: Robustly tuning tail latencies of cloud systems},
     author={Li, Zhao Lucis and Liang, Chieh-Jan Mike and He, Wenjia and Zhu, Lianjie and Dai, Wenjun and Jiang, Jin and Sun, Guangzhong},
     booktitle={2018 $\{$USENIX$\}$ Annual Technical Conference ($\{$USENIX$\}$$\{$ATC$\}$ 18)},
     pages={981--992},
     year={2018}
   }


* `OpEvo: An Evolutionary Method for Tensor Operator Optimization <https://arxiv.org/abs/2006.05664>`__

.. code-block:: bibtex

   @article{Gao2021opevo, 
        title={OpEvo: An Evolutionary Method for Tensor Operator Optimization}, 
        volume={35},
        url={https://ojs.aaai.org/index.php/AAAI/article/view/17462}, 
        number={14}, 
        journal={Proceedings of the AAAI Conference on Artificial Intelligence},
        author={Gao, Xiaotian and Cui, Wei and Zhang, Lintao and Yang, Mao},
        year={2021}, month={May}, pages={12320-12327}
   }

Measurement and Understanding
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


* `Deeper insights into weight sharing in neural architecture search <https://arxiv.org/pdf/2001.01431.pdf>`__

.. code-block:: bibtex

   @article{zhang2020deeper,
     title={Deeper insights into weight sharing in neural architecture search},
     author={Zhang, Yuge and Lin, Zejun and Jiang, Junyang and Zhang, Quanlu and Wang, Yujing and Xue, Hui and Zhang, Chen and Yang, Yaming},
     journal={arXiv preprint arXiv:2001.01431},
     year={2020}
   }


* `How Does Supernet Help in Neural Architecture Search? <https://arxiv.org/abs/2010.08219>`__

.. code-block:: bibtex

   @article{zhang2020does,
     title={How Does Supernet Help in Neural Architecture Search?},
     author={Zhang, Yuge and Zhang, Quanlu and Yang, Yaming},
     journal={arXiv preprint arXiv:2010.08219},
     year={2020}
   }

Applications
^^^^^^^^^^^^


* `AutoADR: Automatic Model Design for Ad Relevance <https://arxiv.org/pdf/2010.07075.pdf>`__

.. code-block:: bibtex

   @inproceedings{chen2020autoadr,
     title={AutoADR: Automatic Model Design for Ad Relevance},
     author={Chen, Yiren and Yang, Yaming and Sun, Hong and Wang, Yujing and Xu, Yu and Shen, Wei and Zhou, Rong and Tong, Yunhai and Bai, Jing and Zhang, Ruofei},
     booktitle={Proceedings of the 29th ACM International Conference on Information \& Knowledge Management},
     pages={2365--2372},
     year={2020}
   }