BERT-NAR-BERT: A Non-Autoregressive Pre-Trained Sequence-to-Sequence Model Leveraging BERT Checkpoints
We introduce BERT-NAR-BERT (BnB) – a pre-trained non-autoregressive sequence-to-sequence model, which employs BERT as the backbone for the encoder and decoder for natural language understanding and generation tasks. During the pre-training and fine-tuning with BERT-NAR-BERT, two challenging aspects are considered by adopting the length classification and connectionist temporal classification models to control the output length of BnB. We evaluate it using a standard natural language understanding benchmark GLUE and three generation tasks – abstractive summarization, question generation, and machine translation. Our results show substantial improvements in inference speed (on average 10x faster) with only little deficiency in output quality when compared to our direct autoregressive baseline BERT2BERT model. Our code is publicly released on GitHub ( https://github.com/aistairc/BERT-NAR-BERT ) under the Apache 2.0 License.
View this article on IEEE Xplore
Cross Domain Early Crop Mapping Using CropSTGAN
Driven by abundant satellite imagery, machine learning-based approaches have recently been promoted to generate high-resolution crop cultivation maps to support many agricultural applications. One of the major challenges faced by these approaches is the limited availability of ground truth labels. In the absence of ground truth, existing work usually adopts the “direct transfer strategy” that trains a classifier using historical labels collected from other regions and then applies the trained model to the target region. Unfortunately, the spectral features of crops exhibit inter-region and inter-annual variability due to changes in soil composition, climate conditions, and crop progress, the resultant models perform poorly on new and unseen regions or years. Despite recent efforts, such as the application of the deep adaptation neural network (DANN) model structure in the deep adaptation crop classification network (DACCN), to tackle the above cross-domain challenges, their effectiveness diminishes significantly when there is a large dissimilarity between the source and target regions. This paper introduces the Crop Mapping Spectral-temporal Generative Adversarial Neural Network (CropSTGAN), a novel solution for cross-domain challenges, that doesn’t require target domain labels. CropSTGAN learns to transform the target domain’s spectral features to those of the source domain, effectively bridging large dissimilarities. Additionally, it employs an identity loss to maintain the intrinsic local structure of the data. Comprehensive experiments across various regions and years demonstrate the benefits and effectiveness of the proposed approach. In experiments, CropSTGAN is benchmarked against various state-of-the-art (SOTA) methods. Notably, CropSTGAN significantly outperforms these methods in scenarios with large data distribution dissimilarities between the target and source domains.
View this article on IEEE Xplore
A Comprehensive Study of Laboratory-Based Micro-CT for 3D Virtual Histology of Human FFPE Tissue Blocks
Advances in laboratory-based X-ray computed tomography (CT) have enabled X-ray 3D virtual histology. This method shows a great potential as a complementary technique to conventional 2D histology where extensive volumetric sampling is necessary. While formalin-fixed paraffin-embedded (FFPE) tissue blocks are the backbone of clinical histology, there exists no generic optimization, and technical study of the X-ray 3D virtual histology of FFPE blocks. X-ray micro-CT of FFPE blocks is studied and optimized in their native state within the cassette to minimize the interference of X-ray 3D virtual histology with clinical workflows and standards, hence facilitating the technology transfer to the clinics. The optimization is carried on the sample positioning, tungsten tubes acceleration voltage, and artifact reduction. Then propagation-based imaging of FFPE blocks is extensively discussed. Hierarchical (local) tomography and laminography are presented as viable approaches for achieving higher spatial resolutions. In the end, future perspectives are given by considering state-of-the-art micro-CT scanners using liquid-metal-jet sources, large-area detectors, and photon counting detectors. The results achieved here are generic and can be applicable to any laboratory-based scanner with a tungsten target source and cone-beam geometry. This article provides a starting point for anyone new to X-ray 3D virtual histology on FFPE blocks, but also serves as a useful source for more experienced users.
View this article on IEEE Xplore
Predicting Energy Loss and Permeability of Field-Annealed Amorphous and Nanocrystalline Alloys up to 1 GHz
Crucial limitations on the permissible energy dissipation are one main factor hindering the use of soft magnetic cores at high frequencies. However, applications find limited support in present-day empirical magnetic loss models, which can hardly afford seamless high-frequency extrapolation of the predicting tools available at low frequencies. This is the case, for example, of very thin soft magnetic plates and ribbons, where the rise of eddy currents and their shielding effects at high frequencies must be attuned to the rate-dependent magnetic constitutive equation of the material. We provide in this work a comprehensive broadband (DC-1 GHz) magnetic characterization and the associated physical modeling of the energy loss and permeability properties of different types of amorphous and nanocrystalline ribbons,
View this article on IEEE Xplore
Controlling the Skyrmion Density and Size for Quantized Convolutional Neural Network
The exceptional properties of skyrmion devices, including their miniature size, topologically protected nature, and low current requirements, render them highly promising for energy-efficient neuromorphic computing applications. Examining the creation, stability, and dynamics of magnetic skyrmions in thin-film systems is imperative to realize these skyrmion-based neuromorphic devices. Herein, we report the creation, stability, and tunability of magnetic skyrmions in the Ta/IrMn/CoFeB/MgO thin-film system. We use polar magneto-optic Kerr effect (MOKE) microscopy and micromagnetic simulations to investigate the magnetic-field dependence of skyrmion density and size. The topological charge evolution with time under a magnetic field is studied, and the transformation dynamics are explained. Furthermore, we demonstrate skyrmion size and density tunability as parameters controlled by voltage, current, and magnetic field via Voltage-Controlled Magnetoresistance (VCMA) and Dzyaloshinskii-Moriya Interaction (DMI). We propose a skyrmion-based synaptic device for neuromorphic computing applications. The device exhibits spin-orbit torque-controlled discrete topological resistance states with high linearity and uniformity, allowing for the realization of the hardware implementation of weight quantization in a Quantized Convolutional Neural Network (QCNN). Our experimental results demonstrate that the devices can be trained and tested on the CIFAR-10 dataset, achieving a recognition accuracy of ~87%. The findings open new avenues for developing neuromorphic computing devices based on tunable skyrmion systems.
View this article on IEEE Xplore
Data Integration for Digital Twins in Industrial Automation: A Systematic Literature Review
The domain of industrial automation faces challenges, such as shortened product life cycles, shortage of skilled labor, and increased complexity. Addressing these issues necessitates innovative solutions, one of which is the Digital Twin, being a virtual counterpart of a physical asset. Central to the quality of a Digital Twin is the data it harnesses. While current Digital Twins primarly draw data from their corresponding physical assets, future interconnected production environments promise an influx of additional data from external devices. However, it remains uncertain how existing Digital Twins incorporate and leverage such data. In this systematic literature review, drawing from a pool of 1107 unique publications, we analyzed 141 works to shed light on data utilization in industrial Digital Twins. We categorized these publications based on Digital Twin types and classified them according to various criteria regarding different characteristics of data. Our findings reveal that the majority of Digital Twins predominantly rely on structured data sourced directly from their associated assets, often employing proprietary integration methods. Facing the trends towards agile and interconnected production ecosystems, as well as an increasing amount of unstructured data, we assert that current Digital Twins are not equipped to meet forthcoming demands in the industrial domain. Consequently, we propose necessary adaptations to fully unleash the potential of Digital Twins and outline future research fields, including automated data integration and evaluation.
View this article on IEEE Xplore
A Lightweight Authentication Scheme for Power IoT Based on PUF and Chebyshev Chaotic Map
With the wide application of IoT technologies in the power sector, power IoT faces serious security challenges, which can be severely affected by malicious attacks and unauthorised access. Meanwhile, devices in power IoT are usually resource-constrained and deployed in a decentralised manner, making them vulnerable to physical attacks. Therefore, a robust and reliable lightweight authentication scheme needs to be constructed to guarantee its information security. A lightweight authentication scheme for the power IoT based on Physical Unclonable Function (PUF) and Chebyshev chaotic map is proposed in this paper, which achieves two-way authentication and session key negotiation between gateways and terminal devices. Comparing with traditional authentication schemes, the PUF and Chebyshev chaotic map used in this scheme have high security and lower resource overhead. PUF is used to generate Challenge and Response Pairs (CRPs) for two-way authentication and key negotiation without storing any secret information about authentication in the device memory. At the same time, Chebyshev chaotic map is used to protect the transmission of secret information such as CRPs in insecure channels. The solution is therefore resistant to attacks such as physical, machine learning modelling and impersonation, ensuring the information security of the authentication process. The proposed scheme is analyzed and verified using the formal verification tool ProVerif and improved BAN logic along with informal methods. The verification results show that the scheme satisfies 12 security properties such as two-way authentication and user anonymity. Comparative analysis with existing related authentication schemes shows that the proposed scheme has low computation and communication costs while guaranteeing security, thus rendering it suitable for resource-constrained terminal devices in the power IoT.
View this article on IEEE Xplore
Extraction of Meta-Data for Recommendation Using Keyword Mapping
Expanding traditional video metadata and recommendation systems encompasses challenges that are difficult to address with conventional methodologies. Limitations in utilizing diverse information when extracting video metadata, along with persistent issues like bias, cold start problems, and the filter bubble effect in recommendation systems, are primary causes of performance degradation. Therefore, a new recommendation system that integrates high-quality video metadata extraction with existing recommendation systems is necessary. This research proposes the “Extraction of Meta-Data for Recommendation using keyword mapping,” which involves constructing contextualized data through object detection models and STT (Speech-to-Text) models, extracting keywords, mapping with the public dataset MovieLens, and applying a Hybrid recommendation system. The process of building contextualized data utilizes YOLO and Google’s Speech-to-Text API. Following this, keywords are extracted using the TextRank algorithm and mapped to the MovieLens dataset. Finally, it is applied to a Hybrid Recommendation System. This paper validates the superiority of this approach by comparing it with the performance of the MovieLens recommendation system that does not expand metadata. Additionally, the effectiveness of metadata expansion is demonstrated through performance comparisons with existing deep learning-based keyword extraction models. Ultimately, this research resolves the cold start and long-tail problems of existing recommendation systems through the construction of video metadata and keyword extraction.
View this article on IEEE Xplore
MCQGen: A Large Language Model-Driven MCQ Generator for Personalized Learning
In the dynamic landscape of contemporary education, the evolution of teaching strategies such as blended learning and flipped classrooms has highlighted the need for efficient and effective generation of multiple-choice questions (MCQs). To address this, we introduce MCQGen, a novel generative artificial intelligence framework designed for the automated creation of MCQs. MCQGen uniquely integrates a large language model (LLM) with retrieval-augmented generation and advanced prompt engineering techniques, drawing from an extensive external knowledge base. This integration significantly enhances the ability of the LLM to produce educationally relevant questions that align with both the goals of educators and the diverse learning needs of students. The framework employs innovative prompt engineering, combining chain-of-thought and self-refine prompting techniques, to enhance the performance of the LLM. This process leads to the generation of questions that are not only contextually relevant and challenging but also reflective of common student misconceptions, contributing effectively to personalized learning experiences and enhancing student engagement and understanding. Our extensive evaluations showcase the effectiveness of MCQGen in producing high-quality MCQs for various educational needs and learning styles. The framework demonstrates its potential to significantly reduce the time and expertise required for MCQ creation, marking its practical utility in modern education. In essence, MCQGen offers an innovative and robust solution for the automated generation of MCQs, enhancing personalized learning in the digital era.
View this article on IEEE Xplore
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Intelligent applications supported by Machine Learning have achieved remarkable performance rates for a wide range of tasks in many domains. However, understanding why a trained algorithm makes a particular decision remains problematic. Given the growing interest in the application of learning-based models, some concerns arise in the dealing with sensible environments, which may impact users’ lives. The complex nature of those models’ decision mechanisms makes them the so-called “black boxes,” in which the understanding of the logic behind automated decision-making processes by humans is not trivial. Furthermore, the reasoning that leads a model to provide a specific prediction can be more important than performance metrics, which introduces a trade-off between interpretability and model accuracy. Explaining intelligent computer decisions can be regarded as a way to justify their reliability and establish trust. In this sense, explanations are critical tools that verify predictions to discover errors and biases previously hidden within the models’ complex structures, opening up vast possibilities for more responsible applications. In this review, we provide theoretical foundations of Explainable Artificial Intelligence (XAI), clarifying diffuse definitions and identifying research objectives, challenges, and future research lines related to turning opaque machine learning outputs into more transparent decisions. We also present a careful overview of the state-of-the-art explainability approaches, with a particular analysis of methods based on feature importance, such as the well-known LIME and SHAP. As a result, we highlight practical applications of the successful use of XAI.
View this article on IEEE Xplore
Follow us: