https://fuldok.hebis.de/xmlui/handle/fuldok/18 2026-05-01T21:31:23Z https://fuldok.hebis.de/xmlui/handle/fuldok/708 Multi-Modal Hand Gesture Recognition using Machine Learning Schak, Monika Mit der steigenden Relevanz von Mensch-Maschinen-Interaktion rückt die Handgestenerkennung immer mehr in das Zentrum der Aufmerksamkeit. Handgesten besitzen ein breites Anwendungsfeld, wie zum Beispiel in der Robotersteuerung oder in Anwendungen der virtuellen Realität. Durch die größere Verfügbarkeit kleiner und günstiger Sensoren, sowie immer größer werdender Rechenleistung spielt auch Multi-Modalität eine immer größere Rolle. Handgesten können damit nicht nur von einem einzigen Sensor, sondern von mehreren unterschiedlichen Sensoren gleichzeitig erfasst werden. Dies ermöglicht die Kombination verschiedener Sichtweisen auf eine Geste, um damit die Erkennung robuster und zuverlässiger zu machen. Das Ziel dieser Dissertation ist die Untersuchung der Robustheit heutiger, weit verbreiteter Methoden der Handgestenerkennung. Ein wichtiges Augenmerk wird dabei auf die Möglichkeit der Verbesserung durch die Kombination verschiedener Sensor-Modalitäten, wie z.B. Tiefendaten, Audio, und Beschleunigungsdaten, gelegt. Außerdem wird im Rahmen dieser Dissertation der umfangreiche Multi-Modale Handgesten-Datensatz (MMHGD) vorgestellt, sowie der davon abgeleitete GestureMNIST-Datensatz. Dieser Datensatz ist öffentlich zugänglich und kann somit auch von anderen Forschern für Experimente im Bereich der Sequenzerkennung und der Multi-Modalen Fusion verwendet werden. Die darin vorhandenen Gestenklassen sind so gestaltet, dass eine Kombination verschiedener Sensor-Modalitäten notwendig ist, um eine zuverlässige Gestenerkennung zu erreichen. Umfangreiche Experimente zeigen die Eignung dieses Datensatzes für die Verwendung in diesen Bereichen. Benchmark-Experimente wurden hierfür mit gängigen Sequenzerkennungs-Verfahren durchgeführt, beispielsweise mit Long Short-Term Memory (LSTM) Netzwerken, Convolutional Neural Networks (CNNs) oder Gaussian Mixture Models (GMMs). Dabei wurden vor allem die Genauigkeit der Gestenerkennung und eine frühzeitige Erkennung der Gesten betrachtet. Ein zweiter Aspekt ist die Robustheit der Gestenerkennung. Im Rahmen dieser Dissertation wird gezeigt, dass die weit verbreiteten LSTM Netzwerke anfällig sind für den Effekt des katastrophalen Vergessens. Gleichzeitig sind sie anfällig für eine Variabilität des Anfangs- und Endzeitpunktes der Gesten. Letzteres kann umgangen werden, wenn die Trainingsdaten bereits mit variablen Start- und Endzeitpunkten erweitert werden. Es konnte gezeigt werden, dass hierfür Augmentationsstrategien verwendet werden können, die sich schon bei der Objekterkennung bewährt haben. Außerdem wird der sogenannte Shifted Recognizer-Ansatz vorgestellt, der verwendet werden kann, um eine zuverlässige Sequenzerkennung zu erreichen, auch wenn der Start- und Endzeitpunkt von Sequenzen nicht bekannt ist. Der dritte Aspekt ist die Fusion mehrerer Sensor-Modalitäten. Hier konnte im Rahmen dieser Dissertation in mehreren Beispielen der frühen und intermediären Fusion gezeigt werden, dass dadurch die Leistung von Sequenzerkennungsverfahren verbessert werden kann, vor allem im Rahmen der frühzeitigen Erkennung.; With the increasing relevance of human-machine interaction, hand-gesture recognition also receives more and more attention. Hand gestures have many applications, such as human-robot control or virtual reality applications. Due to the greater availability of small and cheap sensors, as well as ever-increasing computing power, multi-modality is also playing an increasingly important role in today's research. Hand gestures can thus be detected not only by a single sensor but by several different sensors simultaneously. This allows the combination of different perspectives on a gesture to make the detection more robust and reliable. This dissertation aims to investigate the robustness of state-of-the-art methods of hand gesture recognition. A primary focus is placed on the possibility of improvement by combining different sensory modalities, such as depth data, audio, and acceleration data. In addition, an extensive dataset is presented as part of this dissertation: Multi-Modal Hand Gesture Dataset (MMHGD) as well as the GestureMNIST dataset derived from it. This dataset is publicly available and, therefore, can be used by the research community for experiments in the field of sequence recognition and multi-modal fusion. The gesture classes are designed so that a combination of different sensor modalities is necessary to achieve reliable gesture recognition. Extensive experiments show the suitability of this dataset for use in the areas mentioned above. Benchmark experiments were conducted with state-of-the-art sequence classification methods, such as Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNNs), and Gaussian Mixture Models (GMMs). Above all, the accuracy of gesture recognition and ahead-of-time classification was evaluated. A second aspect is the robustness of gesture recognition. This dissertation shows that the commonly used LSTM network is prone to the catastrophic forgetting effect. At the same time, it is vulnerable to the variability of gesture onset and offset. The latter can be improved if gesture onset variability is already available during the training process. It could also be shown that augmentation strategies, commonly used in object detection, can be used for this. In addition, the so-called Shifted Recognizer Approach is presented, which can be used to achieve reliable sequence detection, even if the start and end times of sequences are not known. The third aspect is the fusion of multiple sensory modalities. Here, experiments with several early and intermediate fusion strategies show that multi-modal fusion improves the performance of sequence recognition methods, especially in the context of ahead-of-time classification. 2025-04-15T00:00:00Z https://fuldok.hebis.de/xmlui/handle/fuldok/700 Improving Network Management Tasks using Machine Learning-Assisted Traffic Engineering in Programmable Network Architectures Hardegen, Christoph (Master of Science (M.Sc.) in Applied Computer Science) The management of modern computer networks becomes increasingly complex, which makes it challenging to determine and deploy decision policies that ensure effective network operation. Reasons for this are ever growing architectures with a large amount of edge and intermediate transit networks. While the first ones connect a huge variety of host systems, the second ones can provide multiple paths of either equal or unequal cost to support higher throughputs and guarantee fault-tolerance. Because devices located at the edge offer or access various network services and applications, data exchange and communication between a pair of involved systems is highly diverse. As versatile runtime behavior results in dynamic network traffic profiles that are subject to constant change, reflected patterns and conditions feature a high degree of complexity. Two affected disciplines are network performance and security management. The former covers tasks that aim to optimize the traffic routing and forwarding in a network. The latter includes those to secure an environment against potential threats, whereby its operation is monitored to discover network attacks and apply appropriate mitigation. Methodologies from the field of network traffic engineering can be leveraged to improve management decisions towards efficient traffic steering and reliable analysis. The general process involves network monitoring and measurement, investigation and optimization steps. In this regard, collected network state and traffic data is analyzed to determine suitable runtime policies that are deployed to pursue a desired management objective. In the context of performance management, equal-cost multi-path routing is a widely adopted standard to distribute the load of upcoming network flows over multiple alternative paths. Since the activity time and intensity level of coherent packet streams highly vary, an inefficient load distribution may result in imbalanced path utilization states. In addition, high loads or network congestion on certain paths have a negative impact on the experience of individual flows that are forwarded along these paths. For example, the perceived latency and the achieved throughput rate may thereby be increased and decreased respectively. One direction for improvement is to take dynamic utilization states into account during the path determination process: First, the use of monitored trends reflecting real conditions from the near past is conceivable. Second, estimated trends representing likely conditions for the near future can be considered. Thus, load balancing decisions are enhanced by performing either utilization- or prediction-aware flow routing. As the actual load of emerging network flows is thereby more evenly shared among available paths, closely balanced and thus efficient saturation levels are ensured. In the context of security management, network intrusion detection and prevention methods help to reveal and handle network attacks. Therefore, one option is to track and export data records for coherent network flows and feed collected data as investigation input to differentiate between benign and malicious packet streams. However, commonly deployed intrusion detection systems are independent and perform isolated traffic analysis that is based on just local attack knowledge. Because attack types and scenarios constantly evolve and may be executed in a highly distributed and coordinated manner, local decision knowledge may not be sufficient to achieve an acceptable level of accuracy and reliability. One direction for improvement is to employ collaborative traffic analysis, whereby a group of local environments forms a logical global setting. Each participant shares its local data views or locally extracted decision knowledge and contributes to a combined global perspective. While this mutual sharing process enhances overall attack detection performance, efficient data exchange and processing is of high importance to ensure scalability. At the same time, timely and granular decision outcomes are required, which enables to apply fast and effective reaction to occuring attacks on network flow level. Machine learning can assist both of the aforementioned network management tasks. For example, traffic prediction approaches can be used to estimate load profiles for upcoming network flows or to classify an observed packet stream as either benign or malicious. In general, large amounts of network traffic data that incorporate high feature diversity can be analyzed to discover included data patterns and extract corresponding knowledge. Then, prepared prediction models can either provide decision support or enable complete autonomy. Whereas the former still involves a human operator, the latter is decoupled and relies on independent system operation. Consequently, open- and closed-loop data processing and decision-making cycles are feasible. Since determined management decisions for traffic control highly depend on the quality of traffic prediction results, traffic monitoring methods have to deliver representative and consistent data views that are consumed as suitable analysis input. Due to the advancements around software-defined networking and the evolvement of programmable switches, network environments can be designed, deployed and operated in a highly flexible manner. While fine-grained network monitoring, analysis and control services are enabled, functionality can be located on centralized controller or distributed switch level. Independent single-step or cooperative multi-step approaches can be provisioned in a device's reconfigurable data plane, its open and customizable operating system (local control plane) or in an assigned controller platform (global control plane). As each system tier is associated with different deployment properties, a systematic combination allows to benefit from their individual advantages. Besides flexible capabilities for initial system provisioning, dynamic runtime adaption is possible as well. The integration of recent paradigms from the field of computer networking with selected methodologies from the field of machine learning allows to move towards self-driving network systems. In order to pursue a high-level management objective, a data processing pipeline with successive steps for continuous data collection and analysis plus subsequent policy enforcement can thereby be autonomously performed. While there are multiple sub-systems that are in charge of a particular runtime task, their individual operation and mutual interaction define overall system behavior. In general, the system's effectiveness in terms of decision efficiency and reliability must be constantly reviewed. If misbehavior or a significant deviation from intended system performance is detected, adaptation has to be initiated and appropriate change applied. This thesis contributes approaches that enable or support effective traffic engineering tasks in the areas of network performance and security management. Therefore, advanced network analytics as well as the principles of softwarized and programmable network architectures are considered. Whereas each proposed approach covers distinct aspects on network monitoring, analysis or control, their combination makes up integrated solutions. Regarding network performance management, predictive flow routing and forwarding is pursued to ensure more efficient load balancing in multi-pathing environments. Load profiling on network flow level allows to estimate likely path utilization states that are taken into account to balance path saturation trends over time more closely. Regarding network security management, collaborative flow classification is pursued to improve the decision accuracy and reliability. Besides an enhanced attack detection performance, efficient data processing with scalable and timely analysis outcomes is achieved. 2025-01-31T00:00:00Z https://fuldok.hebis.de/xmlui/handle/fuldok/635 A framework for current signal based bearing fault detection of permanent magnet synchronous motors Wagner, Tobias Permanently excited synchronous motors are the driving components in countless systems and applications. The most common cause of motor failures are the bearings. Data-driven approaches have been used for predictive defect detections since many years, to prevent motors from an unexpected breakdown. In this way, downtime costs can be reduced and maintenance intervals based on actual wear can be realized. Existing approaches are usually based on structure-borne sound sensors that have to be attached externally to the motors. The resulting costs reduce the economic attractiveness and scalability of the solution. Therefore, the focus of this dissertation is on fault detection based on internal motor current signals. Hurdles, arising from the choice of this signal sources, are to be tackled by the developed fault detection framework. By this, an adequate alternative to the use of external sensors is achieved. The core of the framework is the development of a fault detection pipeline, which is to be applicable under expected conditions of real-world applications. The main pillars are data transformation methods derived from expert knowledge of different domains. These are concatenated and parameterized in an automated manner to reduce the human induced bias on the solution generation process. Starting with a review of the state of research, existing research gaps are identified. From this, the research hypothesis and concrete research questions are derived and the general relevance of research is motivated. Subsequently, a conceptual description of the developed framework is given. In contrast to related work, the proposed approach focuses on the abstraction of the motors operating parameters from the pipeline hyperparameters uniquely at training time. This makes reparameterizations in the course of varied motor parameters obsolete, which increases the robustness with respect to real-world use cases. The data used for the validation of the framework was acquired under real-world operating conditions to enable extensive stress tests of the developed pipelines. The results confirm the suitability of the framework in terms of general current based bearing fault detection as well as the intended use cases, regarding the working condition transfers. 2023-05-12T00:00:00Z https://fuldok.hebis.de/xmlui/handle/fuldok/615 Machine Learning for Industrial Process Optimization De Blasi, Stefano Currently, process control in automation technology is mostly regulated by fixed process parameters as a compromise between several identically constructed systems or by plant operators, who are often guided by intuition based on decades of experience. Some operators are not able to pass on their knowledge to the next generation due to societal developments, e.g. academization or increased desire for self-actualization. In contrast, the vision of Smart Factories includes intelligent machining processes that should ultimately lead to self-optimization and adaptation to uncontrollable variables. To consistently implement this vision of self-optimizing machines, a defined quality criterion must be automatically monitored and act as a feedback for continual, autonomous and safe optimization. The term safe refers to the compliance with process quality standards, which must always be maintained. In a very conservative branch such as automation technology, no risks whatsoever are allowed through random experiments for data generation in production operations, since, for example, an unscheduled downtime leads to serious financial losses. Furthermore, machine-driven decisions may at no time pose a threat. Thus, decisions under uncertainty may only be taken where the amount of uncertainty can be considered uncritical. Additionally, industrial applications require a guaranteed real-time capability in terms of reaction to ensure that the actions can be taken in time whenever needed. Since economic aspects are often crucial for decisions in industry, necessary experiments under laboratory conditions, for example, should also be as avoidable as possible, while the effort required for integration into a field application should be as simple as possible. The aim of this work is the scientific investigation of the integration of learning feedback for intelligent decision making in the control of industrial processes. The successful integration enables data-driven process optimization. To get closer to the vision of self-optimizing machines, safe optimization methods for industrial applications on the process level are investigated and developed. Here, considering the given restrictions of the automation industry is critical. This work addresses several fields including technical, algorithmic and conceptual aspects. The algorithmic refinements are essential for enabling a wider use of safe optimization for industrial applications. They allow, e.g., the automatic handling of the majority of hyper-parameters and the solution of complex problems by increased computational efficiency. Furthermore, the trade-off between exploration and exploitation of safe optimization in high-dimensional spaces is improved. To account for changeable states perceived via sensor data, contextual Bayesian optimization is modified so that safety requirements are met and real-time capability is satisfied. A software application for industrial safe optimization is implemented within a real-time capable control to be able to interact with other software modules to reach an intelligent decision. Further contributions cover recommendations regarding technical requirements with focus on edge control devices and the conceptual inclusion of machine learning to industrial process control. To emphasize the application relevance and feasibility of the presented concepts, real world lighthouse projects are realized in the course of this work, indented to reduce skepticism and thus initiate the breakthrough of self-optimizing machines. 2022-09-28T00:00:00Z