Learning-Based Relaxation of Completeness Requirements for Data Entry Forms

Data entry forms use completeness requirements to specify the fields that are required or optional
to fill for collecting necessary information from different types of users. However, because of the evolving nature of software, some required fields may not be applicable for certain types of users anymore.
Nevertheless, they may still be incorrectly marked as required in the form; we call such fields obsolete
required fields. Since obsolete required fields usually have “not-null” validation checks before submitting
the form, users have to enter meaningless values in such fields in order to complete the form submission.
These meaningless values threaten the quality of the filled data, and could negatively affect stakeholders
or learning-based tools that use the data. To avoid users filling meaningless values, existing techniques
usually rely on manually written rules to identify the obsolete required fields and relax their completeness
requirements. However, these techniques are ineffective and costly.
In this paper, we propose LACQUER, a learning-based automated approach for relaxing the completeness requirements of data entry forms. LACQUER builds Bayesian Network models to automatically
learn conditions under which users had to fill meaningless values. To improve its learning ability, LACQUER identifies the cases where a required field is only applicable for a small group of users, and uses
SMOTE, an oversampling technique, to generate more instances on such fields for effectively mining dependencies on them. During the data entry session, LACQUER predicts the completeness requirement
of a target based on the already filled fields and their conditional dependencies in the trained model.
Our experimental results show that LACQUER can accurately relax the completeness requirements
of required fields in data entry forms with precision values ranging between 0.76 and 0.90 on different
datasets. LACQUER can prevent users from filling 20% to 64% of meaningless values, with negative
predictive values (i.e., the ability to correctly predict a field as “optional”) between 0.72 and 0.91.
Furthermore, LACQUER is efficient; it takes at most 839 ms to predict the completeness requirement of
an instance.