Drop Duplicates

This transform drops duplicate rows from the dataset.

tags: [“Data Preparation”, “Cleaning”]

Parameters

The table gives a brief description about each parameter in Drop Duplicates transform.

Name:

The name for the transform. By default, a name is populated. However, you can provide a custom name for the transform.

Input Dataset:

The file name of the input dataset. You can select the dataset that was uploaded from the drop-down list. (Required: True, Multiple: False)

Output Dataset:

The file name with which the output dataset is created after dropping duplicate columns from the input dataset. (Required: True, Multiple: False)

The sample input for this transform looks as below:

The output after running the drop duplicates transform on the dataset appears as below:

How to use it in Notebook

The following is the code snippet you must use in the Jupyter Notebook editor to run the Drop Duplicates transform:

template=TemplateV2.get_template_by('Drop Duplicates')

recipe_Drop_Duplicates= project.addRecipe([car_data, employee_data, temperature_data, only_numeric], name='Drop Duplicates')

transform=Transform()
transform.templateId = template.id
transform.name='Drop Duplicates'
transform.variables = {
'input_dataset':'car',
'output_dataset':'car_drop_duplicates'}
recipe_Drop_Duplicates.add_transform(transform)
recipe_Drop_Duplicates.run()

Requirements

pandas