Includes bibliographical references.

ACM TIST Special Issue on Deep Learning for Spatio-Temporal Data: Part 1 -- emporal Hierarchical Graph Attention Network for Traffic Prediction -- POLLA: Enhancing the Local Structure Awareness in Long Sequence Spatial-temporal Modeling -- A Dynamic Convolutional Neural Network Based Shared-Bike Demand Forecasting Model -- TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution -- TARA-Net: A Fusion Network for Detecting Takeaway Rider Accidents -- PARP: A Parallel Traffic Condition Driven Route Planning Model on Dynamic Road Networks -- Route Optimization via Environment-Aware Deep Network and Reinforcement Learning -- A Traffic-aware Multi-task Learning Model for Estimating Travel Time -- Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach -- Similar Trajectory Search with Spatio-Temporal Deep Representation Learning -- Predicting Human Mobility with Reinforcement-Learning-Based Long-Term Periodicity Modeling -- Classi-Fly: Inferring Aircraft Categories from Open Data -- Causal Discovery with Confounding Cascade Nonlinear Additive Noise Models -- A Survey of AIOps Methods for Failure Management -- Multi-Stage Fusion and Multi-Source Attention Network for Multi-Modal Remote Sensing Image Segmentation.

[Article Title: ACM TIST Special Issue on Deep Learning for Spatio-Temporal Data: Part 1/ Senzhang Wang, Junbo Zhang, Yanjie Fu and Yong Li, p. 67:1-67:3]

Abstract: With the quick development of different position techniques, such as Global Position System (GPS), mobile devices and remote sensing, various spatio-temporal data have been generated nowadays. It is of great importance for many practical applications, including human mobility mining, urban planning, health care, and public safety, to mine valuable knowledges from spatio-temporal data. Recently, deep learning has achieved considerable success in many domains, such as computer vision, natural language processing, and medical analysis. It also gets remarkable performance gains in various spatio-temporal data mining (STDM) tasks, including crowd flow prediction, origin-destination (OD) prediction, and health care. The aim of this special issue is to show the latest research findings and engineering experiences in developing and applying deep learning techniques for spatio-temporal data mining tasks and applications from the perspective of academia and industry.

https://doi.org/10.1145/3495188

[Article Title: Temporal Hierarchical Graph Attention Network for Traffic Prediction/ Ling Huang, Xing-Xing Liu, Shu-Qiang Huang, Chang-Dong Wang, Wei Tu, Jia-Meng Xie, Shuai Tang and Wendi Xie, p. 68:1-68:21]

Abstract: As a critical task in intelligent traffic systems, traffic prediction has received a large amount of attention in the past few decades. The early efforts mainly model traffic prediction as the time-series mining problem, in which the spatial dependence has been largely ignored. As the rapid development of deep learning, some attempts have been made in modeling traffic prediction as the spatio-temporal data mining problem in a road network, in which deep learning techniques can be adopted for modeling the spatial and temporal dependencies simultaneously. Despite the success, the spatial and temporal dependencies are only modeled in a regionless network without considering the underlying hierarchical regional structure of the spatial nodes, which is an important structure naturally existing in the real-world road network. Apart from the challenge of modeling the spatial and temporal dependencies like the existing studies, the extra challenge caused by considering the hierarchical regional structure of the road network lies in simultaneously modeling the spatial and temporal dependencies between nodes and regions and the spatial and temporal dependencies between regions. To this end, this article proposes a new Temporal Hierarchical Graph Attention Network (TH-GAT). The main idea lies in augmenting the original road network into a region-augmented network, in which the hierarchical regional structure can be modeled. Based on the region-augmented network, the region-aware spatial dependence model and the region-aware temporal dependence model can be constructed, which are two main components of the proposed TH-GAT model. In addition, in the region-aware spatial dependence model, the graph attention network is adopted, in which the importance of a node to another node, of a node to a region, of a region to a node, and of a region to another region, can be captured automatically by means of the attention coefficients. Extensive experiments are conducted on two real-world traffic datasets, and the results have confirmed the superiority of the proposed TH-GAT model.

https://doi.org/10.1145/3446430

[Article Title: POLLA: Enhancing the Local Structure Awareness in Long Sequence Spatial-temporal Modeling/ Haoyi Zhou, Hao Peng, Jieqi Peng, Shuai Zhang and Jianxin Li, p. 69:1-69:24]

Abstract: The spatial-temporal modeling on long sequences is of great importance in many real-world applications. Recent studies have shown the potential of applying the self-attention mechanism to improve capturing the complex spatial-temporal dependencies. However, the lack of underlying structure information weakens its general performance on long sequence spatial-temporal problem. To overcome this limitation, we proposed a novel method, named the Proximity-aware Long Sequence Learning framework, and apply it to the spatial-temporal forecasting task. The model substitutes the canonical self-attention by leveraging the proximity-aware attention, which enhances local structure clues in building long-range dependencies with a linear approximation of attention scores. The relief adjacency matrix technique can utilize the historical global graph information for consistent proximity learning. Meanwhile, the reduced decoder allows for fast inference in a non-autoregressive manner. Extensive experiments are conducted on five large-scale datasets, which demonstrate that our method achieves state-of-the-art performance and validates the effectiveness brought by local structure information.

https://doi.org/10.1145/3447987

[Article Title: A Dynamic Convolutional Neural Network Based Shared-Bike Demand Forecasting Model/ Shaojie Qiao, Nan Han, Jianbin Huang, Kun Yue, Rui Mao, Hongping Shu, Qiang He and Xindong Wu, p. 70:1-70:24]

Abstract: Bike-sharing systems are becoming popular and generate a large volume of trajectory data. In a bike-sharing system, users can borrow and return bikes at different stations. In particular, a bike-sharing system will be affected by weather, the time period, and other dynamic factors, which challenges the scheduling of shared bikes. In this article, a new shared-bike demand forecasting model based on dynamic convolutional neural networks, called SDF, is proposed to predict the demand of shared bikes. SDF chooses the most relevant weather features from real weather data by using the Pearson correlation coefficient and transforms them into a two-dimensional dynamic feature matrix, taking into account the states of stations from historical data. The feature information in the matrix is extracted, learned, and trained with a newly proposed dynamic convolutional neural network to predict the demand of shared bikes in a dynamical and intelligent fashion. The phase of parameter update is optimized from three aspects: the loss function, optimization algorithm, and learning rate. Then, an accurate shared-bike demand forecasting model is designed based on the basic idea of minimizing the loss value. By comparing with classical machine learning models, the weight sharing strategy employed by SDF reduces the complexity of the network. It allows a high prediction accuracy to be achieved within a relatively short period of time. Extensive experiments are conducted on real-world bike-sharing datasets to evaluate SDF. The results show that SDF significantly outperforms classical machine learning models in prediction accuracy and efficiency.

https://doi.org/10.1145/3447988

[Article Title: TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution/ Fayaz Ali Dharejo, Farah Deeba, Yuanchun Zhou, Bhagwan Das, Munsif Ali Jatoi, Muhammad Zawish, Yi Du and Xuezhi Wang, p. 71:1-71:20]

Abstract: Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR). However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatio-temporal remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatio-temporal remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.

https://doi.org/10.1145/3456726

[Article Title: TARA-Net: A Fusion Network for Detecting Takeaway Rider Accidents/ Yifan He, Zhao Li, Lei Fu, Anhui Wang, Peng Zhang, Shuigeng Zhou, Ji Zhang and Ting Yu, p. 72:1-72:19]

Abstract: In the emerging business of food delivery, rider traffic accidents raise financial cost and social traffic burden. Although there has been much effort on traffic accident forecasting using temporal-spatial prediction models, none of the existing work studies the problem of detecting the takeaway rider accidents based on food delivery trajectory data. In this article, we aim to detect whether a takeaway rider meets an accident on a certain time period based on trajectories of food delivery and riders’ contextual information. The food delivery data has a heterogeneous information structure and carries contextual information such as weather and delivery history, and trajectory data are collected as a spatial-temporal sequence. In this article, we propose a TakeAway Rider Accident detection fusion network TARA-Net to jointly model these heterogeneous and spatial-temporal sequence data. We utilize the residual network to extract basic contextual information features and take advantage of a transformer encoder to capture trajectory features. These embedding features are concatenated into a pyramidal feed-forward neural network. We jointly train the above three components to combine the benefits of spatial-temporal trajectory data and sparse basic contextual data for early detecting traffic accidents. Furthermore, although traffic accidents rarely happen in food delivery, we propose a sampling mechanism to alleviate the imbalance of samples when training the model. We evaluate the model on a transportation mode classification dataset Geolife and a real-world Ele.me dataset with over 3 million riders. The experimental results show that the proposed model is superior to the state-of-the-art.

https://doi.org/10.1145/3457218

[Article Title: PARP: A Parallel Traffic Condition Driven Route Planning Model on Dynamic Road Networks/ Tianlun Dai, Bohan Li, Ziqiang Yu, Xiangrong Tong, Meng Chen and Gang Chen, p. 73:1-73:24]

Abstract: The problem of route planning on road network is essential to many Location-Based Services (LBSs). Road networks are dynamic in the sense that the weights of the edges in the corresponding graph constantly change over time, representing evolving traffic conditions. Thus, a practical route planning strategy is required to supply the continuous route optimization considering the historic, current, and future traffic condition. However, few existing works comprehensively take into account these various traffic conditions during the route planning. Moreover, the LBSs usually suffer from extensive concurrent route planning requests in rush hours, which imposes a pressing need to handle numerous queries in parallel for reducing the response time of each query. However, this issue is also not involved by most existing solutions. We therefore investigate a parallel traffic condition driven route planning model on a cluster of processors. To embed the future traffic condition into the route planning, we employ a GCN model to periodically predict the travel costs of roads within a specified time period, which facilitates the robustness of the route planning model against the varying traffic condition. To reduce the response time, a Dual-Level Path (DLP) index is proposed to support a parallel route planning algorithm with the filter-and-refine principle. The bottom level of DLP partitions the entire graph into different subgraphs, and the top level is a skeleton graph that consists of all border vertices in all subgraphs. The filter step identifies a global directional path for a given query based on the skeleton graph. In the refine step, the overall route planning for this query is decomposed into multiple sub-optimizations in the subgraphs passed through by the directional path. Since the subgraphs are independently maintained by different processors, the sub-optimizations of extensive queries can be operated in parallel. Finally, extensive evaluations are conducted to confirm the effectiveness and superiority of the proposal.

https://doi.org/10.1145/3459099

[Article Title: Route Optimization via Environment-Aware Deep Network and Reinforcement Learning/ Pengzhan Guo, Keli Xiao, Zeyang Ye and Wei Zhu, p. 74:1-74:21]

Abstract: Vehicle mobility optimization in urban areas is a long-standing problem in smart city and spatial data analysis. Given the complex urban scenario and unpredictable social events, our work focuses on developing a mobile sequential recommendation system to maximize the profitability of vehicle service providers (e.g., taxi drivers). In particular, we treat the dynamic route optimization problem as a long-term sequential decision-making task. A reinforcement-learning framework is proposed to tackle this problem, by integrating a self-check mechanism and a deep neural network for customer pick-up point monitoring. To account for unexpected situations (e.g., the COVID-19 outbreak), our method is designed to be capable of handling related environment changes with a self-adaptive parameter determination mechanism. Based on the yellow taxi data in New York City and vicinity before and after the COVID-19 outbreak, we have conducted comprehensive experiments to evaluate the effectiveness of our method. The results show consistently excellent performance, from hourly to weekly measures, to support the superiority of our method over the state-of-the-art methods (i.e., with more than 98% improvement in terms of the profitability for taxi drivers).

https://doi.org/10.1145/3461645

[Article Title: TAML: A Traffic-aware Multi-task Learning Model for Estimating Travel Time/ Jiajie Xu, Saijun Xu, Rui Zhou, Chengfei Liu, An Liu and Lei Zhao, p. 75:1-75:14]

Abstract: Travel time estimation has been recognized as an important research topic that can find broad applications. Existing approaches aim to explore mobility patterns via trajectory embedding for travel time estimation. Though state-of-the-art methods utilize estimated traffic condition (by explicit features such as average traffic speed) for auxiliary supervision of travel time estimation, they fail to model their mutual influence and result in inaccuracy accordingly. To this end, in this article, we propose an improved traffic-aware model, called TAML, which adopts a multi-task learning network to integrate a travel time estimator and a traffic estimator in a shared space and improves the accuracy of estimation by enhanced representation of traffic condition, such that more meaningful implicit features are fully captured. In TAML, multi-task learning is further applied for travel time estimation in multi-granularities (including road segment, sub-path, and entire path). The multiple loss functions are combined by considering the homoscedastic uncertainty of each task. Extensive experiments on two real trajectory datasets demonstrate the effectiveness of our proposed methods.

https://doi.org/10.1145/3466686

[Article Title: Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach/ Jayant Gupta, Carl Molnar, Yiqun Xie, Joe Knight and Shashi Shekhar, p. 76:1-76:21]

Abstract: Spatial variability is a prominent feature of various geographic phenomena such as climatic zones, USDA plant hardiness zones, and terrestrial habitat types (e.g., forest, grasslands, wetlands, and deserts). However, current deep learning methods follow a spatial-one-size-fits-all (OSFA) approach to train single deep neural network models that do not account for spatial variability. Quantification of spatial variability can be challenging due to the influence of many geophysical factors. In preliminary work, we proposed a spatial variability aware neural network (SVANN-I, formerly called SVANN) approach where weights are a function of location but the neural network architecture is location independent. In this work, we explore a more flexible SVANN-E approach where neural network architecture varies across geographic locations. In addition, we provide a taxonomy of SVANN types and a physics inspired interpretation model. Experiments with aerial imagery based wetland mapping show that SVANN-I outperforms OSFA and SVANN-E performs the best of all.

https://doi.org/10.1145/3466688

[Article Title: Similar Trajectory Search with Spatio-Temporal Deep Representation Learning/ David Alexander Tedjopurnomo, Xiucheng Li, Zhifeng Bao, Gao Cong, Farhana Choudhury and A. K. Qin, p. 77:1-77:26]

Abstract: Similar trajectory search is a crucial task that facilitates many downstream spatial data analytic applications. Despite its importance, many of the current literature focus solely on the trajectory’s spatial similarity while neglecting the temporal information. Additionally, the few papers that use both the spatial and temporal features based their approach on a traditional point-to-point comparison. These methods model the importance of the spatial and temporal aspect of the data with only a single, pre-defined balancing factor for all trajectories, even though the relative spatial and temporal balance can change from trajectory to trajectory. In this article, we propose the first spatio-temporal, deep-representation-learning-based approach to similar trajectory search. Experiments show that utilizing both features offers significant improvements over existing point-to-point comparison and deep-representation-learning approach. We also show that our deep neural network approach is faster and performs more consistently compared to the point-to-point comparison approaches.

https://doi.org/10.1145/3466687

[Article Title: Predicting Human Mobility with Reinforcement-Learning-Based Long-Term Periodicity Modeling/ Shuo Tao, Jingang Jiang, Defu Lian, Kai Zheng and Enhong Chen, p. 78:1-78:23]

Abstract: Mobility prediction plays an important role in a wide range of location-based applications and services. However, there are three problems in the existing literature: (1) explicit high-order interactions of spatio-temporal features are not systemically modeled; (2) most existing algorithms place attention mechanisms on top of recurrent network, so they can not allow for full parallelism and are inferior to self-attention for capturing long-range dependence; (3) most literature does not make good use of long-term historical information and do not effectively model the long-term periodicity of users. To this end, we propose MoveNet and RLMoveNet. MoveNet is a self-attention-based sequential model, predicting each user’s next destination based on her most recent visits and historical trajectory. MoveNet first introduces a cross-based learning framework for modeling feature interactions. With self-attention on both the most recent visits and historical trajectory, MoveNet can use an attention mechanism to capture the user’s long-term regularity in a more efficient way. Based on MoveNet, to model long-term periodicity more effectively, we add the reinforcement learning layer and named RLMoveNet. RLMoveNet regards the human mobility prediction as a reinforcement learning problem, using the reinforcement learning layer as the regularization part to drive the model to pay attention to the behavior with periodic actions, which can help us make the algorithm more effective. We evaluate both of them with three real-world mobility datasets. MoveNet outperforms the state-of-the-art mobility predictor by around 10% in terms of accuracy, and simultaneously achieves faster convergence and over 4x training speedup. Moreover, RLMoveNet achieves higher prediction accuracy than MoveNet, which proves that modeling periodicity explicitly from the perspective of reinforcement learning is more effective.

https://doi.org/10.1145/3469860

[Article Title: Classi-Fly: Inferring Aircraft Categories from Open Data/ Martin Strohmeier, Matthew Smith, Vincent Lenders and Ivan Martinovic, p. 79:1-79:23]

Abstract: In recent years, air traffic communication data has become easy to access, enabling novel research in many fields. Exploiting this new data source, a wide range of applications have emerged, from weather forecasting to stock market prediction, or the collection of intelligence about military and government movements. Typically, these applications require knowledge about the metadata of the aircraft, specifically its operator and the aircraft category.

armasuisse Science + Technology, the R&D agency for the Swiss Armed Forces, has been developing Classi-Fly, a novel approach to obtain metadata about aircraft based on their movement patterns. We validate Classi-Fly using several hundred thousand flights collected through open source means, in conjunction with ground truth from publicly available aircraft registries containing more than 2 million aircraft. We show that we can obtain the correct aircraft category with an accuracy of greater than 88%. In cases, where no metadata is available, this approach can be used to create the data necessary for applications working with air traffic communication. Finally, we show that it is feasible to automatically detect particular sensitive aircraft such as police and surveillance aircraft using this method.

https://doi.org/10.1145/3480969

[Article Title: Causal Discovery with Confounding Cascade Nonlinear Additive Noise Models/ Jie Qiao, Ruichu Cai, Kun Zhang, Zhenjie Zhang and Zhifeng Hao, p. 80:1-80:28]

Abstract: Identification of causal direction between a causal-effect pair from observed data has recently attracted much attention. Various methods based on functional causal models have been proposed to solve this problem, by assuming the causal process satisfies some (structural) constraints and showing that the reverse direction violates such constraints. The nonlinear additive noise model has been demonstrated to be effective for this purpose, but the model class does not allow any confounding or intermediate variables between a cause pair–even if each direct causal relation follows this model. However, omitting the latent causal variables is frequently encountered in practice. After the omission, the model does not necessarily follow the model constraints. As a consequence, the nonlinear additive noise model may fail to correctly discover causal direction. In this work, we propose a confounding cascade nonlinear additive noise model to represent such causal influences–each direct causal relation follows the nonlinear additive noise model but we observe only the initial cause and final effect. We further propose a method to estimate the model, including the unmeasured confounding and intermediate variables, from data under the variational auto-encoder framework. Our theoretical results show that with our model, the causal direction is identifiable under suitable technical conditions on the data generation process. Simulation results illustrate the power of the proposed method in identifying indirect causal relations across various settings, and experimental results on real data suggest that the proposed model and method greatly extend the applicability of causal discovery based on functional causal models in nonlinear cases.

https://doi.org/10.1145/3482879

[Article Title: A Survey of AIOps Methods for Failure Management/ Paolo Notaro, Jorge Cardoso and Michael Gerndt, p. 81:1-81:45]

Abstract: Modern society is increasingly moving toward complex and distributed computing systems. The increase in scale and complexity of these systems challenges O&M teams that perform daily monitoring and repair operations, in contrast with the increasing demand for reliability and scalability of modern applications. For this reason, the study of automated and intelligent monitoring systems has recently sparked much interest across applied IT industry and academia. Artificial Intelligence for IT Operations (AIOps) has been proposed to tackle modern IT administration challenges thanks to Machine Learning, AI, and Big Data. However, AIOps as a research topic is still largely unstructured and unexplored, due to missing conventions in categorizing contributions for their data requirements, target goals, and components. In this work, we focus on AIOps for Failure Management (FM), characterizing and describing 5 different categories and 14 subcategories of contributions, based on their time intervention window and the target problem being solved. We review 100 FM solutions, focusing on applicability requirements and the quantitative results achieved, to facilitate an effective application of AIOps solutions. Finally, we discuss current development problems in the areas covered by AIOps and delineate possible future trends for AI-based failure management.

https://doi.org/10.1145/3483424

[Article Title: Multi-Stage Fusion and Multi-Source Attention Network for Multi-Modal Remote Sensing Image Segmentation/ Jiaqi Zhao, Yong Zhou, Boyu Shi, Jingsong Yang, Di Zhang and Rui Yao, p. 82:1-82:20]

Abstract: With the rapid development of sensor technology, lots of remote sensing data have been collected. It effectively obtains good semantic segmentation performance by extracting feature maps based on multi-modal remote sensing images since extra modal data provides more information. How to make full use of multi-model remote sensing data for semantic segmentation is challenging. Toward this end, we propose a new network called Multi-Stage Fusion and Multi-Source Attention Network ((MS)2-Net) for multi-modal remote sensing data segmentation. The multi-stage fusion module fuses complementary information after calibrating the deviation information by filtering the noise from the multi-modal data. Besides, similar feature points are aggregated by the proposed multi-source attention for enhancing the discriminability of features with different modalities. The proposed model is evaluated on publicly available multi-modal remote sensing data sets, and results demonstrate the effectiveness of the proposed method.

https://doi.org/10.1145/3484440