Global-Local Latent on Learning Model for Temporal Knowledge Graphs Based on Graph Neural Networks

Guanqiao Fu; Shibin Zhang; Zhi Qin; Pengchneg Liu; Min Yang; Yuanyuan Huang

doi:10.63313/CS.8006

Authors

Guanqiao Fu School of Cybersecurity, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author
Shibin Zhang College of Artificial Intelligence, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author
Zhi Qin School of Cybersecurity, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author
Pengchneg Liu School of Cybersecurity, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author
Min Yang School of Cybersecurity, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author
Yuanyuan Huang College of Artificial Intelligence, Xin Gu Industrial College, Chengdu University of Information Technology, Chengdu 610225, China Author

DOI:

https://doi.org/10.63313/CS.8006

Keywords:

Temporal knowledge graph, Graph neutural network, Latent relationship learning

Abstract

Temporal Knowledge Graphs (TKGs) are an effective method for representing inter-entity and inter-event relationships in continuous time. The inference of temporal knowledge graphs refers to the completion of missing information for already existing points in time, and the extrapolation is the prediction of unknowable facts occurring in the future, and these two kinds of reasoning have important practical value in various fields. However, in the reasoning of temporal knowledge graphs, the lack of capturing potential relationships between entities between different times or within the same time when encoding local and global historical information has led to models that often perform poorly in the face of complex political events. To address these issues, this paper proposes a global-local latent relationship learning model for temporal knowledge graphs based on graph neural networks. Specifically, the model employs a latent relationship learning module to mine and exploit latent relationships within a time slice with multiple adjacent time slices to better simulate and interpret local and global historical information, enhancing the model's ability to deal with complex real-world event problems. Experimental results on four benchmark datasets show a 3 to 4 percentage point improvement in the model's inference capability compared to the state-of-the-art inference model.

References

[1] P.Ernst, C.Meng, A.Siu, and G.Weikum, “Knowlife: A knowledge graph for health and life sciences,” in IEEE 30th International Confer- ence on Data Engineering, Chicago, ICDE 2014, IL, USA, March 31 - April 4, 2014, I.F.Cruz, E.Ferrari, Y.Tao, E.Bertino, and G.Trajcevski, Eds. IEEE Computer Society, 2014, pp. 1254–1257.

[2] C.Yang and G.Qi, “An urban traffic knowledge graph-driven spatial-temporal graph convo-lutional network for traffic flow prediction,” in Proceedings of the 11th International Joint Conference on Knowledge Graphs, IJCKG 2022, Hangzhou, China, October 27-28, 2022. ACM, 2022, pp. 110–114.

[3] L.Cui, H.Seo, M.Tabar, F.Ma, S.Wang, and D.Lee, “DETERRENT: knowledge guided graph at-tention network for detecting healthcare misinformation,” in KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020. ACM, 2020, pp. 492–502.

[4] C.Zhu, M.Chen, C.Fan, G.Cheng, and Y.Zhang,“Learning from history:Modeling temporal knowledge graphs with sequential copy-generation networks,” vol.35, no.5, pp.4732–4740,2021,number: 5.

[5] Z.Li, X.Jin, W.Li, S.Guan, J.Guo, H.Shen, Y.Wang, and X.Cheng, “Temporal knowledge graph reasoning based on evolutional representation learning,” in Proceedings of the 44th In-ternational ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2021, pp. 408–417.

[6] Mengqi Zhang, Yuwei Xia, Qiang Liu, Shu Wu, and Liang Wang, “Learning Latent Relations for Temporal Knowledge Graph Reasoning,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics,Association for Computational Linguistics, 2023, pp.1-10.

[7] Y.Li, S.Sun,and J.Zhao,“Tirgn:time-guided recurrent graph network with local-global his-torical patterns for temporal knowledge graph reasoning,” in Proceedings of the Thir-ty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022. ijcai. org, 2022, pp. 2152–2158.

[8] Y.He, P.Zhang, L.Liu, Q.Liang, W.Zhang,and C.Zhang, “HIP network: Historical information passing network for extrapolation reasoning on temporal knowledge graph,” in Proceed-ings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021.ijcai.org, 2021, pp. 1915–1921.

[9] A.Bordes, N.Usunier, A.Garcia-Duran, J.Weston, and O.Yakhnenko,“Translating embeddings for modeling multi-relational data,” in The 27th Annual Conference on Neural Information Processing Systems 2013,December 5-8, 2013, Lake Tahoe, Nevada, United States, 2013, pp.2787–2795.

[10] Z.Wang, J.Zhang, J.Feng, and Z.Chen, “Knowledge graph embedding by translating on hy-perplanes,” in Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Quebec City, Quebec,Canada. AAAI Press, 2014, pp. 1112–1119.

[11] B.Yang, W.Yih, X.He, J.Gao, and L.Deng, “Embedding entities and relations for learning and inference in knowledge bases,” in 3rd International Conference on Learning Representa-tions, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.

[12] T.Trouillon, J.Welbl, S.Riedel, E.Gaussier, and G.Bouchard,“Complex embeddings for simple link prediction,” in Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, ser. JMLR Workshop and Confer-ence Proceedings, vol. 48. JMLR.org, 2016, pp. 2071–2080.

[13] T.Dettmers, P.Minervini, P.Stenetorp, and S.Riedel, “Convolutional 2d knowledge graph embeddings,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelli-gence, (AAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, 2018, pp. 1811–1818.

[14] C.Shang, Y.Tang, J.Huang, J.Bi, X.He, and B.Zhou, “End-to-end structure-aware convolution-al networks for knowledge base completion,” in The Thirty-Third AAAI Conference on Ar-tificial Intelligence, AAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, 2019, pp. 3060–3067.

[15] M.S.Schlichtkrull, T.N.Kipf, P.Bloem, R.van den Berg, I.Titov,and M.Welling, “Modeling rela-tional data with graph convolutional networks,” in The Semantic Web - 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings, ser.Lecture Notes in Computer Science, vol. 10843. Springer, 2018, pp.593–607.

[16] S.Vashishth, S.Sanyal, V.Nitin, and P.P.Talukdar, “Composition-based multi-relational graph convolutional networks,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa,Ethiopia, April 26-30, 2020. OpenReview.net, 2020.

[17] D.Nathani, J.Chauhan, C.Sharma, and M.Kaul, in Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers.Association for Computational Linguistics, 2019, pp. 4710–4723.

[18] J.Leblay and M.W.Chekol, “Deriving validity time in knowledge graph,” in Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018, P.Champin, F.Gandon, M.Lalmas, and P.G.Ipeirotis, Eds. ACM,2018, pp. 1771–1776.

[19] S.S.Dasgupta, S.N.Ray, and P.P.Talukdar, “Hyte: Hyperplanebased temporally aware knowledge graph embedding,” in Proceedings of the 2018 Conference on Empirical Meth-ods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, 2018, pp. 2001–2011.

[20] T.Lacroix, G.Obozinski, and N.Usunier, “Tensor decompositions for temporal knowledge base completion,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.

[21] Y.Xu, J.Ou, H.Xu, and L.Fu, “Temporal knowledge graph reasoning with historical contras-tive learning,” in Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023. AAAI Press, 2023, pp. 4765–4773.

[22] Z.Li, S.Guan, X.Jin, W.Peng, Y.Lyu, Y.Zhu, L.Bai, W.Li, J.Guo, and X. Cheng, “Complex evolu-tional pattern learning for temporal knowledge graph reasoning,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2022, pp. 290–296.

[23] H.Sun, S.Geng, J.Zhong, H.Hu, and K.He, “Graph hawkes transformer for extrapolated rea-soning on temporal knowledge graphs,” in Proceedings of the 2022 Conference on Empir-ical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emir-ates,December 7-11, 2022. Association for Computational Linguistics, 2022, pp. 7481–7493.

[24] Q.Lin, J.Liu, R.Mao, F.Xu, and E.Cambria, “TECHS: temporal logical graph networks for ex-plainable extrapolation reasoning,” in Proceedings of the 61st Annual Meeting of the Asso-ciation for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto,Canada, July 9-14, 2023. Association for Computational Linguistics, 2023, pp. 1281–1293.

[25] Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang, “Learning to drop: Robust graph neural network via topological denoising,” in WSDM, 2021, pages 779–787.

[26] Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He, “Learning discrete structures for graph neural networks,” in ICML, 2019, pages 1972–1982.

[27] Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, Wei Wang,“Robust Graph Representation Learning via Neural Sparsification,” in Proceed-ings of the 37th International Conference on Machine Learning, 2020, pp. 11458-11468

[28] C.Shang, Y.Tang, J.Huang, J.Bi, X.He, and B.Zhou, “End-to-end structure-aware convolution-al networks for knowledge base completion,” in The Thirty-Third AAAI Conference on Ar-tificial Intelligence, AAAI 2019, Honolulu, Hawaii, USA, January 27-February 1, 2019. AAAI Press, 2019, pp. 3060–3067.