IEEE Access Reproducibility Initiative

IEEE Access Reproducibility Initiative

Reproducibility is at the core of solid scientific and technical research. Consistent with its broad mission to advance technology for the benefit of humanity, IEEE is committed to the principles of open science. For more information on IEEE’s open research initiative, please visit the IEEE Author Center.

IEEE Access is committed to enabling reproducible research through transparency, and the availability and potential reuse of code associated with its published research. IEEE Access is piloting a post-publication peer review of code whereby authors of published IEEE Access articles submit code associated with their article for post-publication peer review.  Once peer-reviewed, published articles can earn a reproducibility badge which helps make published code more visible and credible.

IEEE Access offers the following badges:

  • Code Available: The code, including any associated data and documentation, provided by the authors is reasonable and complete and can potentially be used to support reproducibility of the published results.
  • Code Reviewed: The code, including any associated data and documentation, provided by the authors is reasonable and complete, runs to produce the outputs described, and can support reproducibility of the published results.

This call aims to attract authors who have already published in IEEE Access in the past and seek to improve the reproducibility of their published research. Please read the IEEE Access Reproducibility Author Instructions below.

If awarded, the reproducibility badge will appear with the article in the IEEE Xplore digital library. 

IEEE Access Reproducibility Editors:

Manish Parashar, University of Utah

Porfirio Tramontana, Università degli Studi di Napoli Federico II

Information for Reproducibility Reviewers

As a reviewer, you have a crucial role in supporting research integrity in the peer review and publishing process. Please see our website for guidelines on reviewing for IEEE Access, as well as general best practices for reviewers.

Reviewers will assess the details of the research artifact based on the following criteria:

  1. Documentation: Assess whether the description of the artifact is sufficiently documented to enable them to be exercised by readers of the paper. In particular, keep in mind the accessibility of the code, the code dependencies and requirements, the description of the installation and deployment processes, and the description of the experiments that the artifact implements.
  2. Completeness: Check that the artifact includes all the key components described in the article and to what extent the artifact contributes to the reproducibility of the experiments in the article.
  3. Exercisability: Examine whether the submitted artifact includes the scripts and data needed to execute the experiments described in the article, and whether the software can be successfully executed.

Call for Reproducibility Artifacts: Submission Instructions 

We are inviting submissions of reproducibility artifacts from authors who have an article published in or accepted for publication in IEEE Access and have associated code published in a repository that provides persistent DOI and versioning, such as CodeOcean. CodeOcean is a reproducibility platform. It provides a containerized approach to run artifacts on demand within CodeOcean resources. Your submission should primarily consist of your accepted or published IEEE Access article and all elements required to reproduce the experiments in the article. In essence, the artifacts can include software, datasets, environment configuration, mechanized proofs, benchmarks, test suites with scripts, etc. Instructions or documentation describing the contents and how to use them are also required. The submission necessarily should have everything required for our reviewers to compile and execute the code and reproduce the results. A link to the artifact on a hosting platform and the DOI should also be included.

Please submit your computational artifacts and associated documentation to the IEEE Access artifact review system.

Artifact Description and Structure

The artifact description must be included in a README file along with the artifact, and it must include the following aspects:

  1. Artifact Identification: Including (i) the article’s title, (ii) the author’s names and affiliations, and (iii) an abstract describing the main contributions of the article and how the role of the artifact in these contributions. The abstract may include a software architecture or data models and its description to help the reader understand the artifact and a clear description on to what extent the artifact contributes to the reproducibility of the experiments in the article.
  2. Artifact Dependencies and Requirements: Including (i) a description of the hardware resources required, (ii) a description of the operating systems required, (iii) the software libraries needed, (iv) the input dataset needed to execute the code or when the input data is generated, and (v) optionally, any other dependencies or requirements. Best practices to facilitate the understanding of the descriptions indicate that unnecessary dependencies and requirements should be suppressed from the artifact.
  3. Artifact Installation and Deployment Process: Including (i) the process description to install and compile the libraries and the code, and (ii) the process description to deploy the code in the resources. The description of these processes should include an estimation of the installation, compilation, and deployment times. When any of these times exceed what is reasonable, authors should provide some way to alleviate the effort required by the potential recipients of the artifacts. For instance, capsules with the compiled code can be provided, or a simplified input dataset that reduces the overall experimental execution time. On the other hand, best practices indicate that, whenever it is possible, the actual code of software dependencies (libraries) should not be included in the artifact, but scripts should be provided to download them from a repository and perform the installation.
  4. Reproducibility of Experiments: Including (i) a complete description of the experiment workflow that the code can execute, (ii) an estimation of the execution time to execute the experiment workflow, (iii) a complete description of the expected results and an evaluation of them, and most importantly (iv) how the expected results from the experiment workflow relate to the results found in the article. Best practices indicate that, to facilitate the understanding of the scope of the reproducibility, the expected results from the artifact should be in the same format as the ones in the article. For instance, when the results in the article are depicted in a graph figure, ideally, the execution of the code should provide a (similar) figure (there are open-source tools that can be used for that purpose such as gnuplot). It is critical that authors devote their efforts on these aspects of the reproducibility of experiments to minimize the time needed for their understanding and verification.
  5. Other notes: Including other related aspects that can be important and were not addressed in the previous points.

Please see the following article for an example of an IEEE Access code reviewed badge. 

The Code Reproducibility initiative of IEEE Access follows the steps taken by IEEE Transactions on Parallel and Distributed Systems (TPDS) journal for the same and more detailed guidelines can be found by clicking here.

 

Learn More IEEE Access

IEEE Access is a multidisciplinary, online-only, gold fully open access journal, continuously presenting the results of original research or development across all IEEE fields of interest. Supported by article processing charges (APCs), its hallmarks are rapid peer review, a submission-to-publication time of 4 to 6 weeks, and articles that are freely available to all readers.

IEEE Access publishes articles that are of high interest to readers: original, technically correct, and clearly presented. The scope of this journal comprises all IEEE fields of interest, emphasizing applications-oriented and interdisciplinary articles.

Benefits of Publishing:

  • Quickly announce recent developments to a broad audience in only 4 to 6 weeks
  • Submit multidisciplinary articles 
  • Satisfy Open Access (OA) publishing requirements
  • Publish without Page Limits
  • Connect with Readers Through Commenting
  • Integrate Video
  • View Article Usage and Citation Data on IEEE Xplore®

Top 10 Published Articles of IEEE Access

Over the past ten years, IEEE Access has published some of the most groundbreaking research in electrical and electronics engineering and computer science. In celebration of the 10 Year Publishing Anniversary, our Editors have selected the Top 10 articles published in IEEE Access over the last decade based on downloads, citations, and overall impact in IEEE fields of interest.


Millimeter Wave Mobile Communications for 5G Cellular: It Will Work! (Published in 2013)

Authors: Theodore S. Rappaport, Shu Sun, Rimma Mayzus, Hang Zhao, Yaniv Azar, Kevin Wang, George N. Wong, Jocelyn K. Schulz, Mathew Samimi, Felix Gutierrez

Abstract: The global bandwidth shortage facing wireless carriers has motivated the exploration of the underutilized millimeter wave (mm-wave) frequency spectrum for future broadband cellular communication networks. There is, however, little knowledge about cellular mm-wave propagation in densely populated indoor and outdoor environments. Obtaining this information is vital for the design and operation of future fifth generation cellular networks that use the mm-wave spectrum. In this paper, we present the motivation for new mm-wave cellular systems, methodology, and hardware for measurements and offer a variety of measurement results that show 28 and 38 GHz frequencies can be used when employing steerable directional antennas at base stations and mobile devices.


3D Printing for the Rapid Prototyping of Structural Electronics (Published in 2014)

Authors: Eric Macdonald, Rudy Salas, David Espalin, Mireya Perez, Efrain Aguilera, Dan Muse, Ryan B. Wicker

Abstract: In new product development, time to market (TTM) is critical for the success and profitability of next generation products. When these products include sophisticated electronics encased in 3D packaging with complex geometries and intricate detail, TTM can be compromised – resulting in lost opportunity. The use of advanced 3D printing technology enhanced with component placement and electrical interconnect deposition can provide electronic prototypes that now can be rapidly fabricated in comparable time frames as traditional 2D bread-boarded prototypes; however, these 3D prototypes include the advantage of being embedded within more appropriate shapes in order to authentically prototype products earlier in the development cycle. The fabrication freedom offered by 3D printing techniques, such as stereolithography and fused deposition modeling have recently been explored in the context of 3D electronics integration – referred to as 3D structural electronics or 3D printed electronics. Enhanced 3D printing may eventually be employed to manufacture end-use parts and thus offer unit-level customization with local manufacturing; however, until the materials and dimensional accuracies improve (an eventuality), 3D printing technologies can be employed to reduce development times by providing advanced geometrically appropriate electronic prototypes. This paper describes the development process used to design a novelty six-sided gaming die. The die includes a microprocessor and accelerometer, which together detect motion and upon halting, identify the top surface through gravity and illuminate light-emitting diodes for a striking effect. By applying 3D printing of structural electronics to expedite prototyping, the development cycle was reduced from weeks to hours.


C2PS: A Digital Twin Architecture Reference Model for the Cloud-Based Cyber-Physical Systems (Published in 2017)

Authors: Kazi Masudul Alam, Abdulmotaleb El Saddik

Abstract: Cyber-physical system (CPS) is a new trend in the Internet-of-Things related research works, where physical systems act as the sensors to collect real-world information and communicate them to the computation modules (i.e. cyber layer), which further analyze and notify the findings to the corresponding physical systems through a feedback loop. Contemporary researchers recommend integrating cloud technologies in the CPS cyber layer to ensure the scalability of storage, computation, and cross domain communication capabilities. Though there exist a few descriptive models of the cloud-based CPS architecture, it is important to analytically describe the key CPS properties: computation, control, and communication. In this paper, we present a digital twin architecture reference model for the cloud-based CPS, C2PS, where we analytically describe the key properties of the C2PS. The model helps in identifying various degrees of basic and hybrid computation-interaction modes in this paradigm. We have designed C2PS smart interaction controller using a Bayesian belief network, so that the system dynamically considers current contexts. The composition of fuzzy rule base with the Bayes network further enables the system with reconfiguration capability. We also describe analytically, how C2PS subsystem communications can generate even more complex system-of-systems. Later, we present a telematics-based prototype driving assistance application for the vehicular domain of C2PS, VCPS, to demonstrate the efficacy of the architecture reference model.


Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (Published in 2018)

Authors: Amina Adadi, Mohammed Berrada

Abstract: At the dawn of the fourth industrial revolution, we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our daily life, which contributes to accelerating the shift towards a more algorithmic society. However, even with such unprecedented advancements, a key impediment to the use of AI-based systems is that they often lack transparency. Indeed, the black-box nature of these systems allows powerful predictions, but it cannot be directly explained. This issue has triggered a new debate on explainable AI (XAI). A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption. This survey provides an entry point for interested researchers and practitioners to learn key aspects of the young and rapidly growing body of research related to XAI. Through the lens of the literature, we review the existing approaches regarding the topic, discuss trends surrounding its sphere, and present major research trajectories.


5G Evolution: A View on 5G Cellular Technology Beyond 3GPP Release 15 (Published in 2019)

Authors: Amitabha Ghosh, Andreas Maeder, Matthew Baker, Devaki Chandramouli

Abstract: The 5G System is being developed and enhanced to provide unparalleled connectivity to connect everyone and everything, everywhere. The first version of the 5G System, based on the Release 15 (“Rel-15”) version of the specifications developed by 3GPP, comprising the 5G Core (5GC) and 5G New Radio (NR) with 5G User Equipment (UE), is currently being deployed commercially throughout the world both at sub-6 GHz and at mmWave frequencies. Concurrently, the second phase of 5G is being standardized by 3GPP in the Release 16 (“Rel-16”) version of the specifications which will be completed by March 2020. While the main focus of Rel-15 was on enhanced mobile broadband services, the focus of Rel-16 is on new features for URLLC (Ultra-Reliable Low Latency Communication) and Industrial IoT, including Time Sensitive Communication (TSC), enhanced Location Services, and support for Non-Public Networks (NPNs). In addition, some crucial new features, such as NR on unlicensed bands (NR-U), Integrated Access & Backhaul (IAB) and NR Vehicle-to-X (V2X), are also being introduced as part of Rel-16, as well as enhancements for massive MIMO, wireless and wireline convergence, the Service Based Architecture (SBA) and Network Slicing. Finally, the number of use cases, types of connectivity and users, and applications running on top of 5G networks, are all expected to increase dramatically, thus motivating additional security features to counter security threats which are expected to increase in number, scale and variety. In this paper, we discuss the Rel-16 features and provide an outlook towards Rel-17 and beyond, covering both new features and enhancements of existing features. 5G Evolution will focus on three main areas: enhancements to features introduced in Rel-15 and Rel-16, features that are needed for operational enhancements, and new features to further expand the applicability of the 5G System to new markets and use cases.


Wireless Communications Through Reconfigurable Intelligent Surfaces (Published in 2019)

Authors: Ertugrul Basar, Marco Di Renzo, Julien De Rosny, Merouane Debbah, Mohamed-Slim Alouini, Rui Zhang

Abstract: The future of mobile communications looks exciting with the potential new use cases and challenging requirements of future 6th generation (6G) and beyond wireless networks. Since the beginning of the modern era of wireless communications, the propagation medium has been perceived as a randomly behaving entity between the transmitter and the receiver, which degrades the quality of the received signal due to the uncontrollable interactions of the transmitted radio waves with the surrounding objects. The recent advent of reconfigurable intelligent surfaces in wireless communications enables, on the other hand, network operators to control the scattering, reflection, and refraction characteristics of the radio waves, by overcoming the negative effects of natural wireless propagation. Recent results have revealed that reconfigurable intelligent surfaces can effectively control the wavefront, e.g., the phase, amplitude, frequency, and even polarization, of the impinging signals without the need of complex decoding, encoding, and radio frequency processing operations. Motivated by the potential of this emerging technology, the present article is aimed to provide the readers with a detailed overview and historical perspective on state-of-the-art solutions, and to elaborate on the fundamental differences with other technologies, the most important open research issues to tackle, and the reasons why the use of reconfigurable intelligent surfaces necessitates to rethink the communication-theoretic models currently employed in wireless networks. This article also explores theoretical performance limits of reconfigurable intelligent surface-assisted communication systems using mathematical techniques and elaborates on the potential use cases of intelligent surfaces in 6G and beyond wireless networks.

 

Beyond Herd Immunity Against Strategic Attackers (Published in 2020)

Authors: Vilc Queupe, Leandro Pfleger De Aguiar, Daniel Sadoc Menasche, Cabral Lima, Italo Cunha, Eitan Altman, Rachid El-Azouzi, Francesco De Pellegrini, Alberto Avritzer, Michael Grottke

Abstract: Herd immunity, one of the most fundamental concepts in network epidemics, occurs when a large fraction of the population of devices is immune against a virus or malware. The few individuals who have not taken countermeasures against the threat are assumed to have very low chances of infection, as they are indirectly protected by the rest of the devices in the network. Although very fundamental, herd immunity does not account for strategic attackers scanning the network for vulnerable nodes. In face of such attackers, nodes who linger vulnerable in the network become easy targets, compromising cybersecurity. In this paper, we propose an analytical model which allows us to capture the impact of countermeasures against attackers when both endogenous as well as exogenous infections coexist. Using the proposed model, we show that a diverse set of potential attacks produces non-trivial equilibria, some of which go counter to herd immunity; e.g., our model suggests that nodes should adopt countermeasures even when the remainder of the nodes has already decided to do so.


Unsupervised K-Means Clustering Algorithm (Published in 2020)

Authors: Kristina P. Sinaga, Miin-Shen Yang

Abstract: The k-means algorithm is generally the most known and used clustering method. There are various extensions of k-means to be proposed in the literature. Although it is an unsupervised learning to clustering in pattern recognition and machine learning, the k-means algorithm and its extensions are always influenced by initializations with a necessary number of clusters a priori. That is, the k-means algorithm is not exactly an unsupervised clustering method. In this paper, we construct an unsupervised learning schema for the k-means algorithm so that it is free of initializations without parameter selection and can also simultaneously find an optimal number of clusters. That is, we propose a novel unsupervised k-means (U-k-means) clustering algorithm with automatically finding an optimal number of clusters without giving any initialization and parameter selection. The computational complexity of the proposed U-k-means clustering algorithm is also analyzed. Comparisons between the proposed U-k-means and other existing methods are made. Experimental results and comparisons actually demonstrate these good aspects of the proposed U-k-means clustering algorithm.


Enabling Large Intelligent Surfaces With Compressive Sensing and Deep Learning (Published in 2021)

Authors: Abdelrahman Taha, Muhammad Alrabeiah, Ahmed Alkhateeb

Abstract: Employing large intelligent surfaces (LISs) is a promising solution for improving the coverage and rate of future wireless systems. These surfaces comprise massive numbers of nearly-passive elements that interact with the incident signals, for example by reflecting them, in a smart way that improves the wireless system performance. Prior work focused on the design of the LIS reflection matrices assuming full channel knowledge. Estimating these channels at the LIS, however, is a key challenging problem. With the massive number of LIS elements, channel estimation or reflection beam training will be associated with (i) huge training overhead if all the LIS elements are passive (not connected to a baseband) or with (ii) prohibitive hardware complexity and power consumption if all the elements are connected to the baseband through a fully-digital or hybrid analog/digital architecture. This paper proposes efficient solutions for these problems by leveraging tools from compressive sensing and deep learning. First, a novel LIS architecture based on sparse channel sensors is proposed. In this architecture, all the LIS elements are passive except for a few elements that are active (connected to the baseband). We then develop two solutions that design the LIS reflection matrices with negligible training overhead. In the first approach, we leverage compressive sensing tools to construct the channels at all the LIS elements from the channels seen only at the active elements. In the second approach, we develop a deep-learning based solution where the LIS learns how to interact with the incident signal given the channels at the active elements, which represent the state of the environment and transmitter/receiver locations. We show that the achievable rates of the proposed solutions approach the upper bound, which assumes perfect channel knowledge, with negligible training overhead and with only a few active elements, making them promising for future LIS systems.


DNN-Based Indoor Localization Under Limited Dataset Using GANs and Semi-Supervised Learning (Published in 2022)

Authors: Wafa Njima, Ahmad Bazzi, Marwa Chafii

Abstract: Indoor localization techniques based on supervised learning deliver great performance accuracy while maintaining low online complexity. However, such systems require massive amounts of data for offline training, which necessitates costly measurements. The essence of this paper is twofold with the purpose of providing solutions to missing data of different nature: available unlabeled data and missing unlabeled data. In both cases, we rely on a few labeled available data, which is costly yet insufficient to achieve a high localization accuracy. To address the problem of available unlabeled data, a weighted semi-supervised DNN-based indoor localization approach leveraging pseudo-labeling methods in combination with real labeled samples and inexpensive pseudo-labeled samples is proposed in order to boost localization accuracy, while overcoming the high cost of collecting additional labeled data. As for the extreme case of unavailable unlabeled data, we propose an alternative localization system generating fake fingerprints based on generative adversarial networks (GANs) named ’Weighted GAN based indoor localization’. Furthermore, a deep neural network is trained on a mixed dataset containing both real collected and fake produced data samples using a similar weighting technique in order to improve location prediction performance and avoids overfitting. In terms of localization accuracy, our proposed localization approaches outperform conventional supervised localization schemes utilizing the same collection of real labeled samples. We have tested our proposed methods on both simulated data and experimental data from the publicly available UJIIndoorLoc database, which is built to test indoor positioning systems relying on Wi-Fi fingerprints. Results based on experimental data provide the localization accuracy increase compared to the classical supervised learning method using the same set of labeled collected data when using the weighted semi-supervised and the weighted-GAN approaches by $10.11~\%$ and $8.53~\%$ , respectively.

 

2018 IEEE Access Best Multimedia Award Part 2 Winners

IEEE Access would like to congratulate the winners of the 2018 IEEE Access Best Multimedia Award Part 2 and recipients of a $500 USD Amazon gift card for their fine contributions to IEEE Access. The full article entitled, “FPGA Acceleration for Computationally Efficient Symbol-Level Precoding in Multi-User Multi-Antenna Communication Systems” can be found by clicking here.

 

2018 IEEE Access Best Multimedia Award Part 1 Winners

IEEE Access would like to congratulate the winners of the 2018 IEEE Access Best Multimedia Award Part 1 and recipients of a $500 USD Amazon gift card for their fine contributions to IEEE Access. The full article entitled, “Design and Optimization of a Polar Satellite Mission to Complement the Copernicus Systems” can be found by clicking here.

 

2017 IEEE Access Best Multimedia Contest Part 2 Winners

IEEE Access would like to congratulate the winners of the 2017 IEEE Access Best Multimedia Contest Part 2 and recipients of a $500 USD Amazon gift card for their fine contributions to IEEE Access. The full article entitled, “Antenna Diagnostics and Characterization Using Unmanned Aerial Vehicles” can be found by clicking here.

 

2017 IEEE Access Best Multimedia Contest Part 1 Winners

IEEE Access would like to congratulate the winners of the 2017 IEEE Access Best Multimedia Contest Part 1 and recipients of a $500 USD Amazon gift card for their fine contributions to IEEE Access. View the award-winning video and the full article entitled, “Supporting Trembling Hand Typing Using Optical See-Through Mixed Reality” by clicking here: http://bit.ly/2rxaU7i