In this work we develop a new foundational framework for analyzing Riemannian functional data, including intrinsic Riemannian functional principal component analysis (iRFPCA) and intrinsic Riemannian functional linear regression (iRFLR). The key concept in our development is a novel tensor Hilbert space along a curve on the manifold, based on which Karhunen-Loeve expansion for a Riemannian random process is established for the first time. This framework also features a proper comparison of objects from different tensor Hilbert spaces, which paves the way for asymptotic analysis in Riemannian functional data analysis. Built upon intrinsic geometric concepts such as vector field, Levi-Civita connection and parallel transport on Riemannian manifolds, the proposed framework embraces full generality of applications and proper handle of intrinsic geometric concepts. We then provide estimation procedures for iRFPCA and iRFLR that are distinct from their traditional and/or extrinsic counterparts, and investigate their asymptotic properties within the intrinsic geometry. Numerical performance is illustrated by simulated and real examples.