Alemi, A. A., Fischer, I., Dillon, J. V., and Murphy, K. Deep variational information bottleneck. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
Alet, F., Weng, E., Lozano-Pérez, T., and Kaelbling, L. P. Neural relational inference with fast modular meta-learning. In Advances in Neural Information Processing Systems 32 (NeurIPS), pp. 11804-11815, 2019.
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.
Bianchi, F. M., Scardapane, S., Løkse, S., and Jenssen, R. Reservoir computing approaches for representation and classification of multivariate time series. IEEE transactions on neural networks and learning systems, 32(5): 2169-2179, 2020.
Brasó, G. and Leal-Taixé, L. Learning a neural solver for multiple object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6247-6257, 2020.
Cabrera, D., Sancho, F., and Tobar, F. Combining reservoir computing and variational inference for efficient one-class learning on dynamical systems. In 2017 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), pp. 57-62. IEEE, 2017.
Canaday, D., Pomerance, A., and Girvan, M. A meta-learning approach to reservoir computing: Time series prediction with limited data. arXiv preprint arXiv:2110.03722, 2021.
Chen, P., Liu, R., Aihara, K., and Chen, L. Autoreservoir computing for multistep ahead prediction based on the spatiotemporal information transformation. Nature Communications, 11(1):1-15, 2020.
Chen, S., Wang, J., and Li, G. Neural relational inference with efficient message passing mechanisms. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pp. 7055-7063, 2021.
Dempe, S. Bilevel Optimization: Theory, Algorithms, Applications and a Bibliography, pp. 581-672. Springer International Publishing, Cham, 2020a.
Dempe, S. Bilevel optimization: theory, algorithms, applications and a bibliography. In Bilevel optimization, pp. 581-672. Springer, 2020b.
Dong, J., Rafayelyan, M., Krzakala, F., and Gigan, S. Optical reservoir computing using multiple light scattering for chaotic systems prediction. IEEE Journal of Selected Topics in Quantum Electronics, 26(1):1-12, 2019.
Dong, J., Ohana, R., Rafayelyan, M., and Krzakala, F. Reservoir computing meets recurrent kernels and structured transforms. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020.
Franceschi, L., Frasconi, P., Salzo, S., Grazzi, R., and Pontil, M. Bilevel programming for hyperparameter optimization and meta-learning. In International Conference on Machine Learning (ICML), pp. 1568-1577. PMLR, 2018.
Gallicchio, C. and Micheli, A. Architectural and markovian factors of echo state networks. Neural Networks, 24(5): 440-456, 2011. ISSN 0893-6080.
Gallicchio, C. and Micheli, A. Echo state property of deep reservoir computing networks. Cognitive Computation, 9 (3):337-350, 2017.
Gallicchio, C., Micheli, A., and Pedrelli, L. Deep reservoir computing: A critical experimental analysis. Neurocomputing, 268:87-99, 2017.
Gauthier, D. J., Bollt, E., Griffith, A., and Barbosa, W. A. S. Next generation reservoir computing. Nature Communications, 12:5564, September 2021.
Gong, C., Liu, X., and Liu, Q. Automatic and harmless regularization with constrained and lexicographic optimization: A dynamic barrier approach. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. Communications of the ACM, 63(11):139-144, 2020.
Graber, C. and Schwing, A. G. Dynamic neural relational inference for forecasting trajectories. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 4383-4392, 2020.
Grazzi, R., Franceschi, L., Pontil, M., and Salzo, S. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning (ICML), pp. 3748-3758. PMLR, 2020.
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., and Wang, S. Learning dynamic siamese network for visual object tracking. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Ha, S. and Jeong, H. Unraveling hidden interactions in complex systems with deep learning. Scientific Reports, 11(1):1-13, 2021.
Hong, M., Wai, H.-T., Wang, Z., and Yang, Z. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. arXiv preprint arXiv:2007.05170, 2020.
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 148(34): 13, 2001.
Jalalvand, A., De Neve, W., Van de Walle, R., and Martens, J.-P. Towards using reservoir computing networks for noise-robust image recognition. In 2016 International Joint Conference on Neural Networks (IJCNN), pp. 1666-1672, 2016.
Jalalvand, A., Demuynck, K., De Neve, W., and Martens, J.P. On the application of reservoir computing networks for noisy image recognition. Neurocomputing, 277:237-248, 2018.
Javed, S., Danelljan, M., Khan, F. S., Khan, M. H., Felsberg, M., and Matas, J. Visual object tracking with discriminative filters and siamese networks: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Katok, A. and Hasselblatt, B. Introduction to the Modern Theory of Dynamical Systems. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 1995.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 2688-2697. PMLR, 2018.
Kwapień, J. and Drozdz, S. Physical approach to complex systems. Physics Reports, 515(3):115-226, 2012.
Li, J., Ma, H., Zhang, Z., Li, J., and Tomizuka, M. Spatiotemporal graph dual-attention network for multi-agent prediction and tracking. arXiv preprint arXiv:2102.09117, 2021.
Liu, B., Ye, M., Wright, S., Stone, P., and qiang liu. BOME! bilevel optimization made easy: A simple first-order approach. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022a.
Liu, R., Liu, X., Zeng, S., Zhang, J., and Zhang, Y. Value-function-based sequential minimization for bi-level optimization. arXiv preprint arXiv:2110.04974, 2021.
Liu, R., Gao, J., Zhang, J., Meng, D., and Lin, Z. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 (12):10045-10067, 2022b.
Löwe, S., Madras, D., Shilling, R. Z., and Welling, M. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Proceedings of the 1st Conference on Causal Learning and Reasoning (CLeaR), pp. 509-525. PMLR, 2022.
Maass, W., Natschläger, T., and Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation, 14(11):2531-2560, 2002.
Mehra, A. and Hamm, J. Penalty method for inversion-free deep bilevel optimization. In Asian Conference on Machine Learning (ACML), pp. 347-362. PMLR, 2021.
Nakajima, K. and Fischer, I. Reservoir Computing. Springer, 2021.
Outrata, J. V. On the numerical solution of a class of stack-elberg problems. Zeitschrift für Operations Research, 34 (4):255-277, 1990.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems(NeurIPS), volume 32, pp. 8024-8035, 2019.
Pathak, J., Hunt, B., Girvan, M., Lu, Z., and Ott, E. Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120(2):024102, 2018.
Pedregosa, F. Hyperparameter optimization with approximate gradient. In International conference on machine learning (ICML), pp. 737-746. PMLR, 2016.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A., and Murali, T. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature Methods, 17(2):147-154, 2020.
Saelens, W., Cannoodt, R., Todorov, H., and Saeys, Y. A comparison of single-cell trajectory inference methods. Nature Biotechnology, 37(5):547-554, 2019.
Salimans, T., Karpathy, A., Chen, X., and Kingma, D. P. Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017.
Schrauwen, B., Verstraeten, D., and Van Campenhout, J. An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th european symposium on artificial neural networks., pp. 471-482, 2007.
Shao, C. and Feng, Y. Overcoming catastrophic forgetting beyond continual learning: Balanced training for neural machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 2023-2036, Dublin, Ireland, May 2022. Association for Computational Linguistics.
Shwartz-Ziv, R. and Tishby, N. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810, 2017.
Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., and Woolrich, M. W. Network modelling methods for FMRI. Neuroimage, 54(2):875-891, 2011.
Srinivasan, K., Coble, N., Hamlin, J., Antonsen, T., Ott, E., and Girvan, M. Parallel machine learning for forecasting the dynamics of complex networks. Physical Review Letters, 128(16):164101, 2022.
Stackelberg, H. v. et al. Theory of the market economy. 1952.
Suh, S., Chae, D. H., Kang, H.-G., and Choi, S. Echo-state conditional variational autoencoder for anomaly detection. In 2016 International Joint Conference on Neural Networks (IJCNN), pp. 1015-1022. IEEE, 2016.
Tishby, N. and Zaslavsky, N. Deep learning and the information bottleneck principle. In Proceedings of 2015 IEEE Information Theory Workshop (ITW), pp. 1-5. IEEE, 2015.
Tishby, N., Pereira, F., and Biale, W. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 368-377. IEEE, 1999.
Tsubaki, M., Tomii, K., and Sese, J. Compound-protein interaction prediction with end-to-end learning of neural networks for graphs and sequences. Bioinformatics, 35 (2):309-318, 2019.
Van Den Oord, A., Vinyals, O., et al. Neural discrete representation learning. Advances in Neural Information Processing Systems (NIPS), 30, 2017.
Vlachas, P. R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E., and Koumoutsakos, P. Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics. Neural Networks, 126:191-217, 2020.
Wang, A. and Pang, J. Iterative structural inference of directed graphs. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022.
Webb, E., Day, B., Andres-Terre, H., and Lió, P. Factorised neural relational inference for multi-interaction systems. arXiv preprints arXiv:1905.08721, 2019.
Wu, T., Breuel, T., Skuhersky, M., and Kautz, J. Discovering nonlinear relations with minimum predictive information regularization. arXiv preprint arXiv:2001.01885, 2020.
Wyffels, F. and Schrauwen, B. A comparative study of reservoir computing strategies for monthly time series prediction. Neurocomputing, 73(10-12):1958-1964, 2010.
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
Yildiz, I. B., Jaeger, H., and Kiebel, S. J. Re-visiting the echo state property. Neural Networks, 35:1-9, 2012.
Zhang, Z. and Peng, H. Deeper and wider siamese networks for real-time visual tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
