The dynamical behavior of complex systems pervades various disciplines, from engineering to biology, where understanding and accurately modeling the underlying processes are paramount. State space systems, a fundamental framework in control theory and system dynamics, provide a powerful mathematical representation for capturing the evolving states of dynamic systems over time. This master's thesis aims to explore and advance the understanding of such Dynamical Systems, particularly in the context of parameter estimation, with a focus on leveraging the capabilities of Dynamic Neural Networks. Given any LTI system describing the underlying dynamics, we investigate the topology of the Dynamic Neural Network by deriving a mapping between the state space model and the parameters of the neural network. We also study the loss landscape and incorporate our findings into the learning paradigm.