Supported Input parameters for Transform operation

When you perform the transform operation, depending on your cluster environment, you must make sure to configure the env.sh file for the supported input values accordingly.

Input paramaters Description
PYTHON_COMMAND_INVOKER

This configuration is for the python command name or path to the python command file. Python 2.x and 3.x both versions are supported. If python version 3.X command is not aliased with python then python3 must be entered here. If python 2 is installed then python or python2 must be provided as a value to this config.

Default : python

Mandatory: Yes

RANGER_POLICYMIGRATION_EXCLUDE_SERVICE_TYPES

This configuration is to exclude the Ranger policies of given Ranger service types during the transformation process. Ranger policies of given service-types will not be added in the transformed output file.

Example: storm,tag

Default : storm,tag

Mandatory: No

SOURCE_DESTINATION_RANGER_SERVICE_TYPE_MAPPING

This config is to tell the utility the mapping of source and target Ranger service type. Based on the given source service type, Ranger policies will be transformed to a given target service type. Source and target service type are separated by colon (:)

At present, Only Ranger hdfs service policies can be converted to s3 only.

Example: {hdfs:s3}

Default : {hdfs:s3}

Mandatory: No

SOURCE_DESTINATION_RANGER_SERVICE_NAME_MAPPING

This configuration is to provide the mapping of source and target Ranger service names to the utility. Based on the given service name mapping, during transformation service names of given Ranger policies will be replaced with the target service names.

Source and target service names are separated by colon (:). Multiple entries are allowed and need to be separated by comma(,) character. If no mapping is provided then service names will remain the same even after the transformation process.

Example: {cm_hdfs:cm_s3,c1_yarn:cm_yarn}

Default : empty string

Mandatory: No

SOURCE_DESTINATION_RANGER_ZONE_NAME_MAPPING

This configuration is to provide the mapping of source and target Ranger zone names to the utility. Based on the given zone name mapping, during transformation, zone names of given Ranger policies will be replaced with the target zone names.

Source and target zone names are separated by colon (:). Multiple entries are allowed and need to be separated by comma(,) character. If no mapping is provided then zone names will remain the same even after the transformation process.

Example: {oldzone1:newzone1}

Default : empty string

Mandatory: No

RANGER_POLICYMIGRATION_RESOURCE_MAPPING_FILE

This configuration is to provide the source and target service resource mappings to the utility. Based on the given resources mapping, during transformation resources of given Ranger policies will be replaced with the target resources. If no mapping is provided then the resource string will remain the same even after the transformation process.

File format is CSV. Local filesystem path is supported only.

Example value: /path to /resourceMappings.csv

Default : empty string

Mandatory: No

Resource mapping for hdfs policy resource path=> For the JSON file format we can have multiple resource also in the JSON object formatted like :

Sample resource mapping entries for a json file:

{"/srcdir1/srcfile1":"/tardir1/tarfile1",
          "/srcdir2/srcfile2":"/tardir2/tarfile2"}

=> And with the CSV format, it's pipe-delimited with one entry per line.

Sample resource mapping entries for a csv file:

/srcdir1/srcfile1|/tardir1/tarfile1

/srcdir2/srcfile1|/tardir2/tarfile1

RANGER_POLICYMIGRATION_HIVE_URL_MAPPING_FILE

This config is to provide the source and target HIVE service URL policy resource mappings to the utility. Based on the given URL mapping, during transformation the URL resource of given Ranger Hive URL policies will be replaced with the target URL value. If no mapping is provided then the resource string will remain the same even after the transformation process.

File format is CSV. Local filesystem path is supported only.

Example value: /path to /resourceMappings.csv

Default : empty string

Mandatory: No

Resource mapping for hive policy URL resource=> For the JSON file format we can have multiple resource also in the JSON object formatted like :

Sample resource mapping entries for a json file:

{"/user/hive/warehouse/customers/customers":"s3a://mybucket/user/hive/warehouse/customers/customers","hdfs://dysentry01-3.dysentry01.root.hwx.site:8020":"s3a://mybucket1"}

=> And with the CSV format, it's pipe-delimited with one entry per line.

Sample resource mapping entries for a csv file:

/user/hive/warehouse|s3a://mybucket/user/hive/warehouse

hdfs://localhost:8020|s3a://mybucket1

S3_BUCKET_NAME

This configuration is for S3 bucket name which shall be assigned as a value to the resource type “bucket” key of each transformed policy of S3.

If the target Ranger service type is S3 then value for this property must be provided.

Default : myS3Bucket

Mandatory: No

RANGER_POLICYMIGRATION_USERS_MAPPING_FILE

This configuration is to provide the mapping of usernames in the source and target Ranger policies. Based on the given user names mapping, during transformation user names in the given Ranger policies will be replaced with the target user names. If no mapping is provided then user names will remain the same even after the transformation process.

File format is CSV. Local filesystem path is supported only.

Example: /path/to/usersMappings.csv

Default : empty string

Mandatory: No

Format:

=> For the JSON file format we can have multiple users also in the JSON object formatted like :

{“testuser1”:”testuser1@host.com”, “testuser2”:”testuser2@host.com”}

=> And with the CSV format, it's pipe-delimited with one entry per line.

testuser1|testuser1@host.com

testuser2|testuser2@host.com

RANGER_POLICYMIGRATION_TRANSFORM_OUTPUT_FILE

This configuration is for the output file of transformed Ranger policies.

If a location is given then the user must have write permission on that file/directory.

If an empty string is given then the location shall be the same as the location of the input file. Local file system path should be used for this configuration.

If a directory path is given then the suffix “transform” will be added in the input file name. If path contains file name also then utility shall create/overwrite the existing file with the transformed Ranger policies JSON content.

Example : /tmp

Default : empty string

Mandatory: No

SERVICE_NAME_FOR_NATIVE_POLICIES

This configuration is to assign a service name to the HDFS Ranger policies. These Ranger policies are those Ranger policies which are transformed from the native HDFS permissions.

Example : cm_hdfs

Default : cm_hdfs

Mandatory: Yes

RANGER_POLICYMIGRATION_LOG_DIR

Path to log file directory. Transform utility shall write the log files in this directory.

POLICYMIGRATION_INSTALL_DIR

Path of migration utility parents directory, default is present working directory.

Example : ${PWD}

Default : ${PWD}

Mandatory: No

LOGFILE Name of the log file. Transform utility shall write all the logs to this file.
LOGFILES Name of the log file. Presently there is a single log file for all operations of the migration utility.
RANGER_POLICYMIGRATION_MAX_HEAP_SIZE This allocates Max heap size for the transform utility execution. Default value is 4GB.
UNIX_USER This configuration is for the user who will be the owner of Ranger policy migration utilities directory and files.
UNIX_USER_PWD
UNIX_GROUP This configuration is for the group name which shall have the same permissions as the owner of the Ranger policy migration utility directory and files.