Python toolbox for OpenStreetMap data












5














Background



My project is a Python toolbox. This is my first bigger coding project and it took me quite some time to code this toolbox. I learned a lot from the start point to this day. I changed this code hundreds of times, whenever I learned about new trick or method. This code works. I tested it with various datasets, fixed bugs or any syntax errors.



Cons:




  • The code ha some Polish names for: variable, functions etc.

  • GUI is all Polish

  • I started using Python about 3 months ago


What my code does:



Main purpose of this toolbox was to automate OpenStreetMap (OSM) data transformation from voivodeship shapefiles into country sized one, from which values were selected by their attributes to visualize features (for example, roads were selected and symbolized).



The code consists of three classes which are three scripts inside of my toolbox.



It is used in ArcGIS Pro to help non-programmer user to replicate my work.



My goal



Can someone who is more experienced than me in Python give me some useful advice?



Terms used in this code




  • shp - shapefile

  • osm - OpenStreetMap

  • fc - feature class

  • gdb - geodatabase


I added comments to my code to help understand what is happening.



My code



# -*- coding: CP1250 -*-
import arcpy
import os
import pandas as pd
import shutil
import xlrd
from xml.etree import ElementTree as ET
import glob
from itertools import starmap
import re


class Toolbox(object):
def __init__(self):
"""Define the toolbox (the name of the toolbox is the name of the
.pyt file)."""
self.label = "NardzedziaDoEskportu"
self.alias = ""

# List of tool classes associated with this toolbox
self.tools = [Przygotowanie_do_eksportu, SkryptDoEksportu, XML_export]


class SkryptDoEksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "OSM Polska"
self.description = "Skrypt eksportuje wybrane kolumny zawarte w tabeli atrybutow klas obiektow z geobazy."
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

# drugi parametr
klasy = arcpy.Parameter(
displayName="Warstwy w geobazie (mozliwy tylko podglad)",
name="fcs_of_gdb",
datatype="DEFeatureClass",
parameterType="Required",
direction="Input",
multiValue=True)

# trzeci parametr
kolumny = arcpy.Parameter(
displayName="Wybierz kolumny do selekcji",
name="colli",
datatype="GPString",
parameterType="Required",
direction="Input",
multiValue=True)
kolumny.filter.type = "ValueList"

# Czwarty parametr
plikExcel = arcpy.Parameter(
displayName="Plik *.XLS z domenami",
name="excelik",
datatype="DEType",
parameterType="Required",
direction="Input")

# Piaty parametr
plikShpWoj = arcpy.Parameter(
displayName="Plik *.Shp okreslajacy granice wojewodztw",
name="ShpWoj",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

# Szosty parametr
plikBoundary = arcpy.Parameter(
displayName="Plik *.Shp bedacy poprawiona wersja Polska_boundary_ply",
name="shpBoundary",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

p = [inside, klasy, kolumny, plikExcel, plikShpWoj, plikBoundary]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""
parameters[1].enabled = 0

if parameters[0].value:
arcpy.env.workspace = parameters[0].value
fclist = arcpy.ListFeatureClasses()
parameters[1].value = fclist

if parameters[1].value:
fcs = parameters[1].value.exportToString()
single = fcs.split(";")
fields = arcpy.ListFields(single[0])
l1 = [f.name for f in fields]
l2 = ["OBJECTID", "Shape", "OSMID", "osmTags", "osmuser", "osmuid", "osmvisible",
"osmversion", "osmchangeset", "osmtimestamp", "osmMemberOf", "osmSupportingElement",
"osmMembers", " Shape_Length", "Shape_Area", "wayRefCount"]
l3 = [czynnik for czynnik in l1 if czynnik not in l2]
parameters[2].filter.list = l3
return

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""


return

class XML_export(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eksport danych z XML"
self.description = "Skrypt przygotowuje dane i eksportuje wybrane aspkety z XML"
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr

inside = arcpy.Parameter(
displayName = "Wejsciowa geobaza",
name = "in_gdb",
datatype = "DEWorkspace",
parameterType = "Required",
direction = "Input",
multiValue = False)

# drugi parametr

rodzaj = arcpy.Parameter(
displayName = "Wybierz typ geometrii",
name = "geom",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)

rodzaj.filter.type = "ValueList"
rodzaj.filter.list = ['pt','ln','ply']

# trzeci parametr
klasy = arcpy.Parameter(
displayName = "Wybrane klasy",
name = "fcs_of_gdb",
datatype = "DEFeatureClass",
parameterType = "Required",
direction = "Input",
multiValue = True)

# czwarty
wojewodztwa_string = arcpy.Parameter(
displayName = "Wybierz wojewodztwa",
name = "colli",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = True)

wojewodztwa_string.filter.type = "ValueList"


#piaty
warstwa = arcpy.Parameter(
displayName = "Wybierz warstwe",
name = "fl_gdb",
datatype = "GPFeatureLayer",
parameterType = "Required",
direction = "Input")

# szosty
wyrazenie = arcpy.Parameter(
displayName = "Wpisz wyrazenie do selekcji",
name = "expres",
datatype = "GPSQLExpression",
parameterType = "Required",
direction = "Input")

wyrazenie.parameterDependencies = [warstwa.name]

# siodmy
folder_xml = arcpy.Parameter(
displayName = "Wskaz folder gdzie znajduja sie pliki w formacie XML",
name = "XMLdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

# osmy

folder_csv = arcpy.Parameter(
displayName = "Wskaz folder gdzie maja zostac zapisane pliki CSV",
name = "CSVdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

#dziewiaty
kolumny = arcpy.Parameter(
displayName = "Wybierz kolumne",
name = "colli2",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)
kolumny.filter.type = "ValueList"

#dziesiaty
check_1 = arcpy.Parameter(
displayName = "Zaznacz aby dokonac zapisu do CSV (niezalecane odznaczanie)",
name = "check1",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)

check_1.value = True
#jedenasty
check_2 = arcpy.Parameter(
displayName = "Zaznacz aby polaczyc pliki CSV w jeden - odznaczenie spowoduje brak laczenia",
name = "check2",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)



p = [inside, rodzaj, klasy, wojewodztwa_string,
kolumny, warstwa, wyrazenie, folder_xml, folder_csv,
check_1, check_2]


return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""


wejsciowa_gdb = parameters[0]
wybrana_geometria = parameters[1]
lista_klas = parameters[2]
wybor_wojewodztwa = parameters[3]
wybor_kolumny = parameters[4]
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
lista_klas.enabled = 0

arcpy.env.workspace = wejsciowa_gdb.value
fclist = arcpy.ListFeatureClasses()
fc_o_wybranej_geometrii =

wybor = wybrana_geometria.valueAsText

if check_box_wartosc_2 and check_box_wartosc_1 == False:
parameters[0].enabled = 0
parameters[1].enabled = 0
parameters[3].enabled = 0
parameters[4].enabled = 0
parameters[5].enabled = 0
parameters[6].enabled = 0

if check_box_wartosc_1 and check_box_wartosc_2 == False:
parameters[0].enabled = 1
parameters[1].enabled = 1
parameters[3].enabled = 1
parameters[4].enabled = 1
parameters[5].enabled = 1
parameters[6].enabled = 1

for fc in fclist:
try:
split_nazwy = fc.split('_')
if len (split_nazwy) == 2 and split_nazwy[1] == wybor:
fc_o_wybranej_geometrii.append(fc)

except IndexError:
pass


lista_klas.value = fc_o_wybranej_geometrii

if lista_klas.value:
fcs = lista_klas.value.exportToString()
fcs_lista = fcs.split(";")
wybor_wojewodztwa.filter.list = fcs_lista

if wybrana_geometria.value:
if wybor == 'ln':
lista_ln = [
'highway', 'waterway', 'boundary'
]
wybor_kolumny.filter.list = lista_ln

elif wybor == 'pt':
lista_pt = [
'natural', 'aeroway', 'historic',
'leisure', 'waterway', 'shop',
'railway', 'tourism', 'highway',
'amenity'
]
wybor_kolumny.filter.list = lista_pt

elif wybor == 'ply':
lista_ply = [
'landuse', 'building', 'natural',
'amenity'
]
wybor_kolumny.filter.list = lista_ply



def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):
# Zmienne
# -*- coding: CP1250 -*-
arcpy.env.overwriteOutput = True
tymczasowa_nazwa = "tymczasowaNazwaDlaFC"
gdb = parameters[0].valueAsText
user_geometry_choice = parameters[1].valueAsText
user_wojewodztwo_choice = parameters[3].valueAsText
user_column_choice = parameters[4].valueAsText
user_expression = parameters[6].valueAsText
dir_xml = parameters[7].valueAsText
dir_csv = parameters[8].valueAsText
field_osm = 'OSMID'
xml_parent_way = 'way'
xml_parent_node = 'node'
xml_atr_parent = 'id'
xml_child = 'tag'
xml_atr_child = 'k'
xml_value_child_1 = 'name'
xml_value_child_2 = 'v'
xml_value_child_3 = 'ele'
xml_value_child_4 = 'addr:housenumber'
xml_value_child_5 = 'ref'
id_csv = 'id_robocze'
id_csv_2 = 'id_elementu'
nazwa_csv = 'nazwa'
natural_name = "nazwa_ele"
natural_name_2 = "wysokosc"
building_name = "budynki_nazwa"
building_name_2 = "buydnki_numery"
natural_csv_name = 'natural_nazwa'
natural_csv_name_2 = 'natural_wysokosc'
building_csv_name = 'budynki_nazwa'
building_csv_name_2 = 'budynki_numery'
highway_name = 'ulice'
highway_name_2 = 'nr_drogi'
highway_csv_name = 'ulice'
highway_csv_name_2 = 'nr_drogi'
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
dir_natural = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name,
user_geometry_choice))
dir_natural_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name_2,
user_geometry_choice))
dir_any = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(user_column_choice,
user_geometry_choice))
dir_building = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name,
user_geometry_choice))
dir_building_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name_2,
user_geometry_choice))
dir_highway = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name,
user_geometry_choice))
dir_highway_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name_2,
user_geometry_choice))

# Selekcja z geobazy plikow, ktore zostana wykorzystane do stworzenia list fc
if check_box_wartosc_1:
selektor_pre(
gdb, user_geometry_choice, user_wojewodztwo_choice,
user_column_choice, tymczasowa_nazwa, user_expression)

get_csv(
gdb, user_geometry_choice, user_column_choice, field_osm, dir_xml,
xml_parent_node, xml_atr_parent, xml_child, xml_atr_child,
xml_value_child_1, xml_value_child_3, dir_csv, natural_csv_name,
natural_csv_name_2, id_csv, natural_name, natural_name_2,
xml_value_child_4, building_csv_name, building_csv_name_2,
building_name, building_name_2, xml_value_child_2, nazwa_csv,
xml_parent_way, highway_csv_name, highway_csv_name_2,
highway_name, highway_name_2, xml_value_child_5,
user_geometry_choice, user_column_choice, check_box_wartosc_1,
check_box_wartosc_2, id_csv_2, dir_natural, dir_natural_2, dir_any,
dir_building, dir_building_2, dir_highway, dir_highway_2)

return

class Przygotowanie_do_eksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eliminacja datasetow"
self.description = "Skrypt przygotowuje dane w geobazie, aby spelnialy wymagania nastepnego skryptu."
self.canRunInBackground = False


def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

p =[inside]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):

arcpy.env.overwriteOutput = True


arcpy.env.workspace = parameters[0].valueAsText
alt = arcpy.env.workspace
datalist = arcpy.ListDatasets()

#clears gdb out of data sets
for data in datalist:
for fc in arcpy.ListFeatureClasses("*", "ALL", data):
czesc = fc.split("_")
arcpy.FeatureClassToFeatureClass_conversion(
fc, alt, '{0}_{1}'.format(czesc[0], czesc[2]))
arcpy.Delete_management(data)


return

def import_excel(
in_excel, out_gdb):

"""
Opens excel file from path
Make a list from sheets in file
Iterates through sheets
"""
workbook = xlrd.open_workbook(in_excel)
sheets = [sheet.name for sheet in workbook.sheets()]
for sheet in sheets:
out_table = os.path.join(
out_gdb,
arcpy.ValidateTableName(
"{0}".format(sheet),
out_gdb))

arcpy.ExcelToTable_conversion(in_excel, out_table, sheet)

def iter_kolumny(
user_input, tymczasowa_mazwa,
warunek):
"""
Selection based on user choice

"""
lista_kolumn = user_input.split(";")
arcpy.AddMessage(
"Wybrales nastepujace parametry: {0}".format(lista_kolumn))
fc_lista = arcpy.ListFeatureClasses()
for fc in fc_lista:
czlon_nazwy = fc.split("_")
for kolumna in lista_kolumn:
arcpy.MakeFeatureLayer_management(fc, tymczasowa_mazwa)
try:
arcpy.SelectLayerByAttribute_management(
tymczasowa_mazwa, "NEW_SELECTION", '{0}{1}'.format(kolumna, warunek))
arcpy.CopyFeatures_management(
tymczasowa_mazwa, '{0}_{1}_{2}'.format(czlon_nazwy[0], kolumna, czlon_nazwy[1]))
except arcpy.ExecuteError:
pass
arcpy.Delete_management(fc)


def kolumny_split(
user_input, tymczasowa_mazwa, warunek,
gdb, wojewodztwa_shp, boundary_ply):
"""
After iter_kolumny call faulty column is deleted,
and new fc is imported which will be substitute for it

"""
iter_kolumny(
user_input, tymczasowa_mazwa, warunek)
arcpy.Delete_management(
'Polska_boundary_ply')
arcpy.FeatureClassToFeatureClass_conversion(
wojewodztwa_shp, gdb, 'GraniceWojewodztw')
arcpy.FeatureClassToFeatureClass_conversion(
boundary_ply, gdb, 'Polska_boundary_ply')

def listy_append(
listaFc, liniowa, polygon, punkty):
"""
Simple list appender

"""
for fc in listaFc:
czlon_nazwy = fc.split("_")
if czlon_nazwy[1] == "ln":
liniowa.append(fc)
elif czlon_nazwy[1] == "ply":
polygon.append(fc)
elif czlon_nazwy[1] == "pt":
punkty.append(fc)

def nadaj_domene(
work_space, wybor_uzytkownika):

"""
Function firstly makes list out of
user choice, then appends only those fcs which
are in gdb, then applies only domains which are wanted by user
(determined by fc choice)
"""
arcpy.env.workspace = work_space

lista_kolumn = wybor_uzytkownika.split(";")
all_tabele_gdb = arcpy.ListTables()
lista_poprawiona_o_kolumny =

for tabela in all_tabele_gdb:
pierwszy_czlon_nazwy = tabela.split("_")[0]
if pierwszy_czlon_nazwy in lista_kolumn:
lista_poprawiona_o_kolumny.append(tabela)
elif pierwszy_czlon_nazwy == 'man':
lista_poprawiona_o_kolumny.append(tabela)
else:
arcpy.Delete_management(tabela)

for tabela in lista_poprawiona_o_kolumny:
lista_robocza =
lista_robocza.append(tabela)
nazwa_domeny = lista_robocza[0]
arcpy.TableToDomain_management(
tabela, 'CODE', 'DESCRIPTION', work_space, nazwa_domeny, '-', 'REPLACE')
arcpy.Delete_management(tabela)

def selektor_pre(
baza_in, geometria, wojewodztwa,
kolumna, tymczasowa_nazwa, user_expression):

"""
Selects features based on user expression
"""
arcpy.env.workspace = baza_in

fc_lista = wojewodztwa.split(';')
arcpy.AddMessage(fc_lista)
for fc in fc_lista:
arcpy.MakeFeatureLayer_management(
fc, tymczasowa_nazwa)
arcpy.SelectLayerByAttribute_management(
tymczasowa_nazwa, "NEW_SELECTION", user_expression)
arcpy.CopyFeatures_management(
tymczasowa_nazwa, '{0}_{1}'.format(fc, kolumna))
arcpy.AddMessage(
'Seleckja skonczona dla {0}_{1}'.format(fc, kolumna))

def compare_save_to_csv(
gdb, pole_osm, xml_folder,
kolumna, parent,atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
csv_dir, nazwa_pliku, nazwa_id, nazwa_atrybutu,
user_geometry_choice):

"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')
if split[2] == kolumna and split[1] == user_geometry_choice:
czesc_nazwy = split[0]
geom = split[1]
nazwa_pliku = '{0}_{1}'.format(kolumna, geom)

lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
csv_dir, nazwa_id, nazwa_atrybutu,czesc_nazwy)

except IndexError:
pass

def compare_save_to_csv_wyjatek(
gdb, user_geometry_choice, user_column_choice,
pole_osm, xml_folder, kolumna, parent, atrybut_parent,
child, child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, csv_name, csv_name_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2):


"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser_wyjatki
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')

if split[2] == kolumna and split[1] == user_geometry_choice:

czesc_nazwy = split[0]
lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser_wyjatki(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent, atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, child_value_3, sciezka_csv,
csv_name, csv_name_2, nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy)

except IndexError:
pass

def merge_csv(
sciezka_csv, fragment_nazwy, nazwa_csv):
"""
Merges csv in specifed directory based on name scheme
"""
results = pd.DataFrame()
for counter, file in enumerate(glob.glob("{0}*{1}*".format(sciezka_csv, fragment_nazwy))):
name_dataframe = pd.read_csv(
file, usecols=[0, 1],encoding = 'CP1250' )
results = results.append(
name_dataframe)
results.to_csv(
'{0}{1}.csv'.format(sciezka_csv, nazwa_csv), encoding = 'CP1250')


def zapis_do_csv(
lista_1, lista_2, nazwa_1,
nazwa_2, csv_name, katalog,
czesc_nazwy):
"""
Saves to CSV, based on 2 lists.
"""
raw_data = {nazwa_1: lista_1,
nazwa_2: lista_2}
df = pd.DataFrame(raw_data, columns=[nazwa_1, nazwa_2])
df.to_csv(
'{0}{1}_{2}.csv'.format(katalog, czesc_nazwy, csv_name),
index=False, header=True, encoding = 'CP1250')


def xml_parser(
xml, lista_agis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
sciezka_csv, nazwa_id, nazwa_atrybutu,
czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))

zapis_do_csv(
l1, l2, nazwa_id,
nazwa_atrybutu, nazwa_pliku,
sciezka_csv, czesc_nazwy)

arcpy.AddMessage('Zapisalem {0}'.format(nazwa_pliku))
arcpy.AddMessage('Zapsialem tyle id: {0}'.format((len(l1))))
arcpy.AddMessage('Zapsialem tyle nazw: {0}'.format((len(l2))))

def xml_parser_wyjatki(
xml, lista_agis,
parent, atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, nazwa_pliku, nazwa_pliku_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

Added 'elif' for some feature classes that are described
by 2 value tags.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
l3 =
l4 =

for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))
elif sselement.attrib[child_atrybut] == child_value_3:
l3.append(
subelement.get(atrybut_parent))
l4.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))

zapis_do_csv(
l1, l2,
nazwa_id, nazwa_atrybutu,
nazwa_pliku, sciezka_csv, czesc_nazwy)

zapis_do_csv(
l3, l4,
nazwa_id, nazwa_atrybutu_2,
nazwa_pliku_2, sciezka_csv, czesc_nazwy)

def replace_csv(
csv, symbol_1, symbol_2):
'''
Function replace certain symbol to prevent
ArcGIS Pro from crashing during table import.
'''
my_csv_path = csv

with open(my_csv_path, 'r') as f:
my_csv_text = f.read()

find_str = symbol_1
replace_str = symbol_2

csv_str = re.sub(find_str, replace_str, my_csv_text)

with open(my_csv_path, 'w') as f:
f.write(csv_str)


def fix_field(
tabela , nazwa, pole):
"""
Imported tables has got not
valid field with ID. This fix that problem
by creating new on in text type, copying value
and deleting old one.
"""
arcpy.AddField_management(
tabela, nazwa, "TEXT", field_length = 20)
try:
with arcpy.da.UpdateCursor(tabela, [pole,nazwa]) as cursor:
for row in cursor:
row[1] = row[0]
cursor.updateRow(row)
except RuntimeError:
print(row[1])
del row,cursor
arcpy.DeleteField_management(tabela, [pole])


def import_fix_join(
in_table, out_gdb, nazwa,
id_csv, in_fc, field_osm,
pole_to_join):
"""
Imports table to geodatabase
Fixes its column
Join field to feature class.

"""
arcpy.TableToGeodatabase_conversion(
[in_table], out_gdb)

fix_field(
in_table, nazwa, id_csv)

pole = [pole_to_join]

arcpy.env.workspace = out_gdb

arcpy.JoinField_management(
in_fc, field_osm, in_table,
nazwa, pole)


Three scripts in ArcGIS Pro software.



enter image description here



Script number one GUI



enter image description here



Script number two GUI



enter image description here



Script number three GUI



enter image description here










share|improve this question
























  • Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
    – Reinderien
    Nov 20 '18 at 16:19
















5














Background



My project is a Python toolbox. This is my first bigger coding project and it took me quite some time to code this toolbox. I learned a lot from the start point to this day. I changed this code hundreds of times, whenever I learned about new trick or method. This code works. I tested it with various datasets, fixed bugs or any syntax errors.



Cons:




  • The code ha some Polish names for: variable, functions etc.

  • GUI is all Polish

  • I started using Python about 3 months ago


What my code does:



Main purpose of this toolbox was to automate OpenStreetMap (OSM) data transformation from voivodeship shapefiles into country sized one, from which values were selected by their attributes to visualize features (for example, roads were selected and symbolized).



The code consists of three classes which are three scripts inside of my toolbox.



It is used in ArcGIS Pro to help non-programmer user to replicate my work.



My goal



Can someone who is more experienced than me in Python give me some useful advice?



Terms used in this code




  • shp - shapefile

  • osm - OpenStreetMap

  • fc - feature class

  • gdb - geodatabase


I added comments to my code to help understand what is happening.



My code



# -*- coding: CP1250 -*-
import arcpy
import os
import pandas as pd
import shutil
import xlrd
from xml.etree import ElementTree as ET
import glob
from itertools import starmap
import re


class Toolbox(object):
def __init__(self):
"""Define the toolbox (the name of the toolbox is the name of the
.pyt file)."""
self.label = "NardzedziaDoEskportu"
self.alias = ""

# List of tool classes associated with this toolbox
self.tools = [Przygotowanie_do_eksportu, SkryptDoEksportu, XML_export]


class SkryptDoEksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "OSM Polska"
self.description = "Skrypt eksportuje wybrane kolumny zawarte w tabeli atrybutow klas obiektow z geobazy."
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

# drugi parametr
klasy = arcpy.Parameter(
displayName="Warstwy w geobazie (mozliwy tylko podglad)",
name="fcs_of_gdb",
datatype="DEFeatureClass",
parameterType="Required",
direction="Input",
multiValue=True)

# trzeci parametr
kolumny = arcpy.Parameter(
displayName="Wybierz kolumny do selekcji",
name="colli",
datatype="GPString",
parameterType="Required",
direction="Input",
multiValue=True)
kolumny.filter.type = "ValueList"

# Czwarty parametr
plikExcel = arcpy.Parameter(
displayName="Plik *.XLS z domenami",
name="excelik",
datatype="DEType",
parameterType="Required",
direction="Input")

# Piaty parametr
plikShpWoj = arcpy.Parameter(
displayName="Plik *.Shp okreslajacy granice wojewodztw",
name="ShpWoj",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

# Szosty parametr
plikBoundary = arcpy.Parameter(
displayName="Plik *.Shp bedacy poprawiona wersja Polska_boundary_ply",
name="shpBoundary",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

p = [inside, klasy, kolumny, plikExcel, plikShpWoj, plikBoundary]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""
parameters[1].enabled = 0

if parameters[0].value:
arcpy.env.workspace = parameters[0].value
fclist = arcpy.ListFeatureClasses()
parameters[1].value = fclist

if parameters[1].value:
fcs = parameters[1].value.exportToString()
single = fcs.split(";")
fields = arcpy.ListFields(single[0])
l1 = [f.name for f in fields]
l2 = ["OBJECTID", "Shape", "OSMID", "osmTags", "osmuser", "osmuid", "osmvisible",
"osmversion", "osmchangeset", "osmtimestamp", "osmMemberOf", "osmSupportingElement",
"osmMembers", " Shape_Length", "Shape_Area", "wayRefCount"]
l3 = [czynnik for czynnik in l1 if czynnik not in l2]
parameters[2].filter.list = l3
return

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""


return

class XML_export(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eksport danych z XML"
self.description = "Skrypt przygotowuje dane i eksportuje wybrane aspkety z XML"
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr

inside = arcpy.Parameter(
displayName = "Wejsciowa geobaza",
name = "in_gdb",
datatype = "DEWorkspace",
parameterType = "Required",
direction = "Input",
multiValue = False)

# drugi parametr

rodzaj = arcpy.Parameter(
displayName = "Wybierz typ geometrii",
name = "geom",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)

rodzaj.filter.type = "ValueList"
rodzaj.filter.list = ['pt','ln','ply']

# trzeci parametr
klasy = arcpy.Parameter(
displayName = "Wybrane klasy",
name = "fcs_of_gdb",
datatype = "DEFeatureClass",
parameterType = "Required",
direction = "Input",
multiValue = True)

# czwarty
wojewodztwa_string = arcpy.Parameter(
displayName = "Wybierz wojewodztwa",
name = "colli",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = True)

wojewodztwa_string.filter.type = "ValueList"


#piaty
warstwa = arcpy.Parameter(
displayName = "Wybierz warstwe",
name = "fl_gdb",
datatype = "GPFeatureLayer",
parameterType = "Required",
direction = "Input")

# szosty
wyrazenie = arcpy.Parameter(
displayName = "Wpisz wyrazenie do selekcji",
name = "expres",
datatype = "GPSQLExpression",
parameterType = "Required",
direction = "Input")

wyrazenie.parameterDependencies = [warstwa.name]

# siodmy
folder_xml = arcpy.Parameter(
displayName = "Wskaz folder gdzie znajduja sie pliki w formacie XML",
name = "XMLdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

# osmy

folder_csv = arcpy.Parameter(
displayName = "Wskaz folder gdzie maja zostac zapisane pliki CSV",
name = "CSVdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

#dziewiaty
kolumny = arcpy.Parameter(
displayName = "Wybierz kolumne",
name = "colli2",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)
kolumny.filter.type = "ValueList"

#dziesiaty
check_1 = arcpy.Parameter(
displayName = "Zaznacz aby dokonac zapisu do CSV (niezalecane odznaczanie)",
name = "check1",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)

check_1.value = True
#jedenasty
check_2 = arcpy.Parameter(
displayName = "Zaznacz aby polaczyc pliki CSV w jeden - odznaczenie spowoduje brak laczenia",
name = "check2",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)



p = [inside, rodzaj, klasy, wojewodztwa_string,
kolumny, warstwa, wyrazenie, folder_xml, folder_csv,
check_1, check_2]


return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""


wejsciowa_gdb = parameters[0]
wybrana_geometria = parameters[1]
lista_klas = parameters[2]
wybor_wojewodztwa = parameters[3]
wybor_kolumny = parameters[4]
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
lista_klas.enabled = 0

arcpy.env.workspace = wejsciowa_gdb.value
fclist = arcpy.ListFeatureClasses()
fc_o_wybranej_geometrii =

wybor = wybrana_geometria.valueAsText

if check_box_wartosc_2 and check_box_wartosc_1 == False:
parameters[0].enabled = 0
parameters[1].enabled = 0
parameters[3].enabled = 0
parameters[4].enabled = 0
parameters[5].enabled = 0
parameters[6].enabled = 0

if check_box_wartosc_1 and check_box_wartosc_2 == False:
parameters[0].enabled = 1
parameters[1].enabled = 1
parameters[3].enabled = 1
parameters[4].enabled = 1
parameters[5].enabled = 1
parameters[6].enabled = 1

for fc in fclist:
try:
split_nazwy = fc.split('_')
if len (split_nazwy) == 2 and split_nazwy[1] == wybor:
fc_o_wybranej_geometrii.append(fc)

except IndexError:
pass


lista_klas.value = fc_o_wybranej_geometrii

if lista_klas.value:
fcs = lista_klas.value.exportToString()
fcs_lista = fcs.split(";")
wybor_wojewodztwa.filter.list = fcs_lista

if wybrana_geometria.value:
if wybor == 'ln':
lista_ln = [
'highway', 'waterway', 'boundary'
]
wybor_kolumny.filter.list = lista_ln

elif wybor == 'pt':
lista_pt = [
'natural', 'aeroway', 'historic',
'leisure', 'waterway', 'shop',
'railway', 'tourism', 'highway',
'amenity'
]
wybor_kolumny.filter.list = lista_pt

elif wybor == 'ply':
lista_ply = [
'landuse', 'building', 'natural',
'amenity'
]
wybor_kolumny.filter.list = lista_ply



def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):
# Zmienne
# -*- coding: CP1250 -*-
arcpy.env.overwriteOutput = True
tymczasowa_nazwa = "tymczasowaNazwaDlaFC"
gdb = parameters[0].valueAsText
user_geometry_choice = parameters[1].valueAsText
user_wojewodztwo_choice = parameters[3].valueAsText
user_column_choice = parameters[4].valueAsText
user_expression = parameters[6].valueAsText
dir_xml = parameters[7].valueAsText
dir_csv = parameters[8].valueAsText
field_osm = 'OSMID'
xml_parent_way = 'way'
xml_parent_node = 'node'
xml_atr_parent = 'id'
xml_child = 'tag'
xml_atr_child = 'k'
xml_value_child_1 = 'name'
xml_value_child_2 = 'v'
xml_value_child_3 = 'ele'
xml_value_child_4 = 'addr:housenumber'
xml_value_child_5 = 'ref'
id_csv = 'id_robocze'
id_csv_2 = 'id_elementu'
nazwa_csv = 'nazwa'
natural_name = "nazwa_ele"
natural_name_2 = "wysokosc"
building_name = "budynki_nazwa"
building_name_2 = "buydnki_numery"
natural_csv_name = 'natural_nazwa'
natural_csv_name_2 = 'natural_wysokosc'
building_csv_name = 'budynki_nazwa'
building_csv_name_2 = 'budynki_numery'
highway_name = 'ulice'
highway_name_2 = 'nr_drogi'
highway_csv_name = 'ulice'
highway_csv_name_2 = 'nr_drogi'
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
dir_natural = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name,
user_geometry_choice))
dir_natural_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name_2,
user_geometry_choice))
dir_any = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(user_column_choice,
user_geometry_choice))
dir_building = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name,
user_geometry_choice))
dir_building_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name_2,
user_geometry_choice))
dir_highway = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name,
user_geometry_choice))
dir_highway_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name_2,
user_geometry_choice))

# Selekcja z geobazy plikow, ktore zostana wykorzystane do stworzenia list fc
if check_box_wartosc_1:
selektor_pre(
gdb, user_geometry_choice, user_wojewodztwo_choice,
user_column_choice, tymczasowa_nazwa, user_expression)

get_csv(
gdb, user_geometry_choice, user_column_choice, field_osm, dir_xml,
xml_parent_node, xml_atr_parent, xml_child, xml_atr_child,
xml_value_child_1, xml_value_child_3, dir_csv, natural_csv_name,
natural_csv_name_2, id_csv, natural_name, natural_name_2,
xml_value_child_4, building_csv_name, building_csv_name_2,
building_name, building_name_2, xml_value_child_2, nazwa_csv,
xml_parent_way, highway_csv_name, highway_csv_name_2,
highway_name, highway_name_2, xml_value_child_5,
user_geometry_choice, user_column_choice, check_box_wartosc_1,
check_box_wartosc_2, id_csv_2, dir_natural, dir_natural_2, dir_any,
dir_building, dir_building_2, dir_highway, dir_highway_2)

return

class Przygotowanie_do_eksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eliminacja datasetow"
self.description = "Skrypt przygotowuje dane w geobazie, aby spelnialy wymagania nastepnego skryptu."
self.canRunInBackground = False


def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

p =[inside]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):

arcpy.env.overwriteOutput = True


arcpy.env.workspace = parameters[0].valueAsText
alt = arcpy.env.workspace
datalist = arcpy.ListDatasets()

#clears gdb out of data sets
for data in datalist:
for fc in arcpy.ListFeatureClasses("*", "ALL", data):
czesc = fc.split("_")
arcpy.FeatureClassToFeatureClass_conversion(
fc, alt, '{0}_{1}'.format(czesc[0], czesc[2]))
arcpy.Delete_management(data)


return

def import_excel(
in_excel, out_gdb):

"""
Opens excel file from path
Make a list from sheets in file
Iterates through sheets
"""
workbook = xlrd.open_workbook(in_excel)
sheets = [sheet.name for sheet in workbook.sheets()]
for sheet in sheets:
out_table = os.path.join(
out_gdb,
arcpy.ValidateTableName(
"{0}".format(sheet),
out_gdb))

arcpy.ExcelToTable_conversion(in_excel, out_table, sheet)

def iter_kolumny(
user_input, tymczasowa_mazwa,
warunek):
"""
Selection based on user choice

"""
lista_kolumn = user_input.split(";")
arcpy.AddMessage(
"Wybrales nastepujace parametry: {0}".format(lista_kolumn))
fc_lista = arcpy.ListFeatureClasses()
for fc in fc_lista:
czlon_nazwy = fc.split("_")
for kolumna in lista_kolumn:
arcpy.MakeFeatureLayer_management(fc, tymczasowa_mazwa)
try:
arcpy.SelectLayerByAttribute_management(
tymczasowa_mazwa, "NEW_SELECTION", '{0}{1}'.format(kolumna, warunek))
arcpy.CopyFeatures_management(
tymczasowa_mazwa, '{0}_{1}_{2}'.format(czlon_nazwy[0], kolumna, czlon_nazwy[1]))
except arcpy.ExecuteError:
pass
arcpy.Delete_management(fc)


def kolumny_split(
user_input, tymczasowa_mazwa, warunek,
gdb, wojewodztwa_shp, boundary_ply):
"""
After iter_kolumny call faulty column is deleted,
and new fc is imported which will be substitute for it

"""
iter_kolumny(
user_input, tymczasowa_mazwa, warunek)
arcpy.Delete_management(
'Polska_boundary_ply')
arcpy.FeatureClassToFeatureClass_conversion(
wojewodztwa_shp, gdb, 'GraniceWojewodztw')
arcpy.FeatureClassToFeatureClass_conversion(
boundary_ply, gdb, 'Polska_boundary_ply')

def listy_append(
listaFc, liniowa, polygon, punkty):
"""
Simple list appender

"""
for fc in listaFc:
czlon_nazwy = fc.split("_")
if czlon_nazwy[1] == "ln":
liniowa.append(fc)
elif czlon_nazwy[1] == "ply":
polygon.append(fc)
elif czlon_nazwy[1] == "pt":
punkty.append(fc)

def nadaj_domene(
work_space, wybor_uzytkownika):

"""
Function firstly makes list out of
user choice, then appends only those fcs which
are in gdb, then applies only domains which are wanted by user
(determined by fc choice)
"""
arcpy.env.workspace = work_space

lista_kolumn = wybor_uzytkownika.split(";")
all_tabele_gdb = arcpy.ListTables()
lista_poprawiona_o_kolumny =

for tabela in all_tabele_gdb:
pierwszy_czlon_nazwy = tabela.split("_")[0]
if pierwszy_czlon_nazwy in lista_kolumn:
lista_poprawiona_o_kolumny.append(tabela)
elif pierwszy_czlon_nazwy == 'man':
lista_poprawiona_o_kolumny.append(tabela)
else:
arcpy.Delete_management(tabela)

for tabela in lista_poprawiona_o_kolumny:
lista_robocza =
lista_robocza.append(tabela)
nazwa_domeny = lista_robocza[0]
arcpy.TableToDomain_management(
tabela, 'CODE', 'DESCRIPTION', work_space, nazwa_domeny, '-', 'REPLACE')
arcpy.Delete_management(tabela)

def selektor_pre(
baza_in, geometria, wojewodztwa,
kolumna, tymczasowa_nazwa, user_expression):

"""
Selects features based on user expression
"""
arcpy.env.workspace = baza_in

fc_lista = wojewodztwa.split(';')
arcpy.AddMessage(fc_lista)
for fc in fc_lista:
arcpy.MakeFeatureLayer_management(
fc, tymczasowa_nazwa)
arcpy.SelectLayerByAttribute_management(
tymczasowa_nazwa, "NEW_SELECTION", user_expression)
arcpy.CopyFeatures_management(
tymczasowa_nazwa, '{0}_{1}'.format(fc, kolumna))
arcpy.AddMessage(
'Seleckja skonczona dla {0}_{1}'.format(fc, kolumna))

def compare_save_to_csv(
gdb, pole_osm, xml_folder,
kolumna, parent,atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
csv_dir, nazwa_pliku, nazwa_id, nazwa_atrybutu,
user_geometry_choice):

"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')
if split[2] == kolumna and split[1] == user_geometry_choice:
czesc_nazwy = split[0]
geom = split[1]
nazwa_pliku = '{0}_{1}'.format(kolumna, geom)

lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
csv_dir, nazwa_id, nazwa_atrybutu,czesc_nazwy)

except IndexError:
pass

def compare_save_to_csv_wyjatek(
gdb, user_geometry_choice, user_column_choice,
pole_osm, xml_folder, kolumna, parent, atrybut_parent,
child, child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, csv_name, csv_name_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2):


"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser_wyjatki
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')

if split[2] == kolumna and split[1] == user_geometry_choice:

czesc_nazwy = split[0]
lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser_wyjatki(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent, atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, child_value_3, sciezka_csv,
csv_name, csv_name_2, nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy)

except IndexError:
pass

def merge_csv(
sciezka_csv, fragment_nazwy, nazwa_csv):
"""
Merges csv in specifed directory based on name scheme
"""
results = pd.DataFrame()
for counter, file in enumerate(glob.glob("{0}*{1}*".format(sciezka_csv, fragment_nazwy))):
name_dataframe = pd.read_csv(
file, usecols=[0, 1],encoding = 'CP1250' )
results = results.append(
name_dataframe)
results.to_csv(
'{0}{1}.csv'.format(sciezka_csv, nazwa_csv), encoding = 'CP1250')


def zapis_do_csv(
lista_1, lista_2, nazwa_1,
nazwa_2, csv_name, katalog,
czesc_nazwy):
"""
Saves to CSV, based on 2 lists.
"""
raw_data = {nazwa_1: lista_1,
nazwa_2: lista_2}
df = pd.DataFrame(raw_data, columns=[nazwa_1, nazwa_2])
df.to_csv(
'{0}{1}_{2}.csv'.format(katalog, czesc_nazwy, csv_name),
index=False, header=True, encoding = 'CP1250')


def xml_parser(
xml, lista_agis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
sciezka_csv, nazwa_id, nazwa_atrybutu,
czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))

zapis_do_csv(
l1, l2, nazwa_id,
nazwa_atrybutu, nazwa_pliku,
sciezka_csv, czesc_nazwy)

arcpy.AddMessage('Zapisalem {0}'.format(nazwa_pliku))
arcpy.AddMessage('Zapsialem tyle id: {0}'.format((len(l1))))
arcpy.AddMessage('Zapsialem tyle nazw: {0}'.format((len(l2))))

def xml_parser_wyjatki(
xml, lista_agis,
parent, atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, nazwa_pliku, nazwa_pliku_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

Added 'elif' for some feature classes that are described
by 2 value tags.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
l3 =
l4 =

for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))
elif sselement.attrib[child_atrybut] == child_value_3:
l3.append(
subelement.get(atrybut_parent))
l4.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))

zapis_do_csv(
l1, l2,
nazwa_id, nazwa_atrybutu,
nazwa_pliku, sciezka_csv, czesc_nazwy)

zapis_do_csv(
l3, l4,
nazwa_id, nazwa_atrybutu_2,
nazwa_pliku_2, sciezka_csv, czesc_nazwy)

def replace_csv(
csv, symbol_1, symbol_2):
'''
Function replace certain symbol to prevent
ArcGIS Pro from crashing during table import.
'''
my_csv_path = csv

with open(my_csv_path, 'r') as f:
my_csv_text = f.read()

find_str = symbol_1
replace_str = symbol_2

csv_str = re.sub(find_str, replace_str, my_csv_text)

with open(my_csv_path, 'w') as f:
f.write(csv_str)


def fix_field(
tabela , nazwa, pole):
"""
Imported tables has got not
valid field with ID. This fix that problem
by creating new on in text type, copying value
and deleting old one.
"""
arcpy.AddField_management(
tabela, nazwa, "TEXT", field_length = 20)
try:
with arcpy.da.UpdateCursor(tabela, [pole,nazwa]) as cursor:
for row in cursor:
row[1] = row[0]
cursor.updateRow(row)
except RuntimeError:
print(row[1])
del row,cursor
arcpy.DeleteField_management(tabela, [pole])


def import_fix_join(
in_table, out_gdb, nazwa,
id_csv, in_fc, field_osm,
pole_to_join):
"""
Imports table to geodatabase
Fixes its column
Join field to feature class.

"""
arcpy.TableToGeodatabase_conversion(
[in_table], out_gdb)

fix_field(
in_table, nazwa, id_csv)

pole = [pole_to_join]

arcpy.env.workspace = out_gdb

arcpy.JoinField_management(
in_fc, field_osm, in_table,
nazwa, pole)


Three scripts in ArcGIS Pro software.



enter image description here



Script number one GUI



enter image description here



Script number two GUI



enter image description here



Script number three GUI



enter image description here










share|improve this question
























  • Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
    – Reinderien
    Nov 20 '18 at 16:19














5












5








5







Background



My project is a Python toolbox. This is my first bigger coding project and it took me quite some time to code this toolbox. I learned a lot from the start point to this day. I changed this code hundreds of times, whenever I learned about new trick or method. This code works. I tested it with various datasets, fixed bugs or any syntax errors.



Cons:




  • The code ha some Polish names for: variable, functions etc.

  • GUI is all Polish

  • I started using Python about 3 months ago


What my code does:



Main purpose of this toolbox was to automate OpenStreetMap (OSM) data transformation from voivodeship shapefiles into country sized one, from which values were selected by their attributes to visualize features (for example, roads were selected and symbolized).



The code consists of three classes which are three scripts inside of my toolbox.



It is used in ArcGIS Pro to help non-programmer user to replicate my work.



My goal



Can someone who is more experienced than me in Python give me some useful advice?



Terms used in this code




  • shp - shapefile

  • osm - OpenStreetMap

  • fc - feature class

  • gdb - geodatabase


I added comments to my code to help understand what is happening.



My code



# -*- coding: CP1250 -*-
import arcpy
import os
import pandas as pd
import shutil
import xlrd
from xml.etree import ElementTree as ET
import glob
from itertools import starmap
import re


class Toolbox(object):
def __init__(self):
"""Define the toolbox (the name of the toolbox is the name of the
.pyt file)."""
self.label = "NardzedziaDoEskportu"
self.alias = ""

# List of tool classes associated with this toolbox
self.tools = [Przygotowanie_do_eksportu, SkryptDoEksportu, XML_export]


class SkryptDoEksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "OSM Polska"
self.description = "Skrypt eksportuje wybrane kolumny zawarte w tabeli atrybutow klas obiektow z geobazy."
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

# drugi parametr
klasy = arcpy.Parameter(
displayName="Warstwy w geobazie (mozliwy tylko podglad)",
name="fcs_of_gdb",
datatype="DEFeatureClass",
parameterType="Required",
direction="Input",
multiValue=True)

# trzeci parametr
kolumny = arcpy.Parameter(
displayName="Wybierz kolumny do selekcji",
name="colli",
datatype="GPString",
parameterType="Required",
direction="Input",
multiValue=True)
kolumny.filter.type = "ValueList"

# Czwarty parametr
plikExcel = arcpy.Parameter(
displayName="Plik *.XLS z domenami",
name="excelik",
datatype="DEType",
parameterType="Required",
direction="Input")

# Piaty parametr
plikShpWoj = arcpy.Parameter(
displayName="Plik *.Shp okreslajacy granice wojewodztw",
name="ShpWoj",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

# Szosty parametr
plikBoundary = arcpy.Parameter(
displayName="Plik *.Shp bedacy poprawiona wersja Polska_boundary_ply",
name="shpBoundary",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

p = [inside, klasy, kolumny, plikExcel, plikShpWoj, plikBoundary]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""
parameters[1].enabled = 0

if parameters[0].value:
arcpy.env.workspace = parameters[0].value
fclist = arcpy.ListFeatureClasses()
parameters[1].value = fclist

if parameters[1].value:
fcs = parameters[1].value.exportToString()
single = fcs.split(";")
fields = arcpy.ListFields(single[0])
l1 = [f.name for f in fields]
l2 = ["OBJECTID", "Shape", "OSMID", "osmTags", "osmuser", "osmuid", "osmvisible",
"osmversion", "osmchangeset", "osmtimestamp", "osmMemberOf", "osmSupportingElement",
"osmMembers", " Shape_Length", "Shape_Area", "wayRefCount"]
l3 = [czynnik for czynnik in l1 if czynnik not in l2]
parameters[2].filter.list = l3
return

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""


return

class XML_export(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eksport danych z XML"
self.description = "Skrypt przygotowuje dane i eksportuje wybrane aspkety z XML"
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr

inside = arcpy.Parameter(
displayName = "Wejsciowa geobaza",
name = "in_gdb",
datatype = "DEWorkspace",
parameterType = "Required",
direction = "Input",
multiValue = False)

# drugi parametr

rodzaj = arcpy.Parameter(
displayName = "Wybierz typ geometrii",
name = "geom",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)

rodzaj.filter.type = "ValueList"
rodzaj.filter.list = ['pt','ln','ply']

# trzeci parametr
klasy = arcpy.Parameter(
displayName = "Wybrane klasy",
name = "fcs_of_gdb",
datatype = "DEFeatureClass",
parameterType = "Required",
direction = "Input",
multiValue = True)

# czwarty
wojewodztwa_string = arcpy.Parameter(
displayName = "Wybierz wojewodztwa",
name = "colli",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = True)

wojewodztwa_string.filter.type = "ValueList"


#piaty
warstwa = arcpy.Parameter(
displayName = "Wybierz warstwe",
name = "fl_gdb",
datatype = "GPFeatureLayer",
parameterType = "Required",
direction = "Input")

# szosty
wyrazenie = arcpy.Parameter(
displayName = "Wpisz wyrazenie do selekcji",
name = "expres",
datatype = "GPSQLExpression",
parameterType = "Required",
direction = "Input")

wyrazenie.parameterDependencies = [warstwa.name]

# siodmy
folder_xml = arcpy.Parameter(
displayName = "Wskaz folder gdzie znajduja sie pliki w formacie XML",
name = "XMLdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

# osmy

folder_csv = arcpy.Parameter(
displayName = "Wskaz folder gdzie maja zostac zapisane pliki CSV",
name = "CSVdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

#dziewiaty
kolumny = arcpy.Parameter(
displayName = "Wybierz kolumne",
name = "colli2",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)
kolumny.filter.type = "ValueList"

#dziesiaty
check_1 = arcpy.Parameter(
displayName = "Zaznacz aby dokonac zapisu do CSV (niezalecane odznaczanie)",
name = "check1",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)

check_1.value = True
#jedenasty
check_2 = arcpy.Parameter(
displayName = "Zaznacz aby polaczyc pliki CSV w jeden - odznaczenie spowoduje brak laczenia",
name = "check2",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)



p = [inside, rodzaj, klasy, wojewodztwa_string,
kolumny, warstwa, wyrazenie, folder_xml, folder_csv,
check_1, check_2]


return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""


wejsciowa_gdb = parameters[0]
wybrana_geometria = parameters[1]
lista_klas = parameters[2]
wybor_wojewodztwa = parameters[3]
wybor_kolumny = parameters[4]
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
lista_klas.enabled = 0

arcpy.env.workspace = wejsciowa_gdb.value
fclist = arcpy.ListFeatureClasses()
fc_o_wybranej_geometrii =

wybor = wybrana_geometria.valueAsText

if check_box_wartosc_2 and check_box_wartosc_1 == False:
parameters[0].enabled = 0
parameters[1].enabled = 0
parameters[3].enabled = 0
parameters[4].enabled = 0
parameters[5].enabled = 0
parameters[6].enabled = 0

if check_box_wartosc_1 and check_box_wartosc_2 == False:
parameters[0].enabled = 1
parameters[1].enabled = 1
parameters[3].enabled = 1
parameters[4].enabled = 1
parameters[5].enabled = 1
parameters[6].enabled = 1

for fc in fclist:
try:
split_nazwy = fc.split('_')
if len (split_nazwy) == 2 and split_nazwy[1] == wybor:
fc_o_wybranej_geometrii.append(fc)

except IndexError:
pass


lista_klas.value = fc_o_wybranej_geometrii

if lista_klas.value:
fcs = lista_klas.value.exportToString()
fcs_lista = fcs.split(";")
wybor_wojewodztwa.filter.list = fcs_lista

if wybrana_geometria.value:
if wybor == 'ln':
lista_ln = [
'highway', 'waterway', 'boundary'
]
wybor_kolumny.filter.list = lista_ln

elif wybor == 'pt':
lista_pt = [
'natural', 'aeroway', 'historic',
'leisure', 'waterway', 'shop',
'railway', 'tourism', 'highway',
'amenity'
]
wybor_kolumny.filter.list = lista_pt

elif wybor == 'ply':
lista_ply = [
'landuse', 'building', 'natural',
'amenity'
]
wybor_kolumny.filter.list = lista_ply



def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):
# Zmienne
# -*- coding: CP1250 -*-
arcpy.env.overwriteOutput = True
tymczasowa_nazwa = "tymczasowaNazwaDlaFC"
gdb = parameters[0].valueAsText
user_geometry_choice = parameters[1].valueAsText
user_wojewodztwo_choice = parameters[3].valueAsText
user_column_choice = parameters[4].valueAsText
user_expression = parameters[6].valueAsText
dir_xml = parameters[7].valueAsText
dir_csv = parameters[8].valueAsText
field_osm = 'OSMID'
xml_parent_way = 'way'
xml_parent_node = 'node'
xml_atr_parent = 'id'
xml_child = 'tag'
xml_atr_child = 'k'
xml_value_child_1 = 'name'
xml_value_child_2 = 'v'
xml_value_child_3 = 'ele'
xml_value_child_4 = 'addr:housenumber'
xml_value_child_5 = 'ref'
id_csv = 'id_robocze'
id_csv_2 = 'id_elementu'
nazwa_csv = 'nazwa'
natural_name = "nazwa_ele"
natural_name_2 = "wysokosc"
building_name = "budynki_nazwa"
building_name_2 = "buydnki_numery"
natural_csv_name = 'natural_nazwa'
natural_csv_name_2 = 'natural_wysokosc'
building_csv_name = 'budynki_nazwa'
building_csv_name_2 = 'budynki_numery'
highway_name = 'ulice'
highway_name_2 = 'nr_drogi'
highway_csv_name = 'ulice'
highway_csv_name_2 = 'nr_drogi'
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
dir_natural = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name,
user_geometry_choice))
dir_natural_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name_2,
user_geometry_choice))
dir_any = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(user_column_choice,
user_geometry_choice))
dir_building = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name,
user_geometry_choice))
dir_building_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name_2,
user_geometry_choice))
dir_highway = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name,
user_geometry_choice))
dir_highway_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name_2,
user_geometry_choice))

# Selekcja z geobazy plikow, ktore zostana wykorzystane do stworzenia list fc
if check_box_wartosc_1:
selektor_pre(
gdb, user_geometry_choice, user_wojewodztwo_choice,
user_column_choice, tymczasowa_nazwa, user_expression)

get_csv(
gdb, user_geometry_choice, user_column_choice, field_osm, dir_xml,
xml_parent_node, xml_atr_parent, xml_child, xml_atr_child,
xml_value_child_1, xml_value_child_3, dir_csv, natural_csv_name,
natural_csv_name_2, id_csv, natural_name, natural_name_2,
xml_value_child_4, building_csv_name, building_csv_name_2,
building_name, building_name_2, xml_value_child_2, nazwa_csv,
xml_parent_way, highway_csv_name, highway_csv_name_2,
highway_name, highway_name_2, xml_value_child_5,
user_geometry_choice, user_column_choice, check_box_wartosc_1,
check_box_wartosc_2, id_csv_2, dir_natural, dir_natural_2, dir_any,
dir_building, dir_building_2, dir_highway, dir_highway_2)

return

class Przygotowanie_do_eksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eliminacja datasetow"
self.description = "Skrypt przygotowuje dane w geobazie, aby spelnialy wymagania nastepnego skryptu."
self.canRunInBackground = False


def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

p =[inside]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):

arcpy.env.overwriteOutput = True


arcpy.env.workspace = parameters[0].valueAsText
alt = arcpy.env.workspace
datalist = arcpy.ListDatasets()

#clears gdb out of data sets
for data in datalist:
for fc in arcpy.ListFeatureClasses("*", "ALL", data):
czesc = fc.split("_")
arcpy.FeatureClassToFeatureClass_conversion(
fc, alt, '{0}_{1}'.format(czesc[0], czesc[2]))
arcpy.Delete_management(data)


return

def import_excel(
in_excel, out_gdb):

"""
Opens excel file from path
Make a list from sheets in file
Iterates through sheets
"""
workbook = xlrd.open_workbook(in_excel)
sheets = [sheet.name for sheet in workbook.sheets()]
for sheet in sheets:
out_table = os.path.join(
out_gdb,
arcpy.ValidateTableName(
"{0}".format(sheet),
out_gdb))

arcpy.ExcelToTable_conversion(in_excel, out_table, sheet)

def iter_kolumny(
user_input, tymczasowa_mazwa,
warunek):
"""
Selection based on user choice

"""
lista_kolumn = user_input.split(";")
arcpy.AddMessage(
"Wybrales nastepujace parametry: {0}".format(lista_kolumn))
fc_lista = arcpy.ListFeatureClasses()
for fc in fc_lista:
czlon_nazwy = fc.split("_")
for kolumna in lista_kolumn:
arcpy.MakeFeatureLayer_management(fc, tymczasowa_mazwa)
try:
arcpy.SelectLayerByAttribute_management(
tymczasowa_mazwa, "NEW_SELECTION", '{0}{1}'.format(kolumna, warunek))
arcpy.CopyFeatures_management(
tymczasowa_mazwa, '{0}_{1}_{2}'.format(czlon_nazwy[0], kolumna, czlon_nazwy[1]))
except arcpy.ExecuteError:
pass
arcpy.Delete_management(fc)


def kolumny_split(
user_input, tymczasowa_mazwa, warunek,
gdb, wojewodztwa_shp, boundary_ply):
"""
After iter_kolumny call faulty column is deleted,
and new fc is imported which will be substitute for it

"""
iter_kolumny(
user_input, tymczasowa_mazwa, warunek)
arcpy.Delete_management(
'Polska_boundary_ply')
arcpy.FeatureClassToFeatureClass_conversion(
wojewodztwa_shp, gdb, 'GraniceWojewodztw')
arcpy.FeatureClassToFeatureClass_conversion(
boundary_ply, gdb, 'Polska_boundary_ply')

def listy_append(
listaFc, liniowa, polygon, punkty):
"""
Simple list appender

"""
for fc in listaFc:
czlon_nazwy = fc.split("_")
if czlon_nazwy[1] == "ln":
liniowa.append(fc)
elif czlon_nazwy[1] == "ply":
polygon.append(fc)
elif czlon_nazwy[1] == "pt":
punkty.append(fc)

def nadaj_domene(
work_space, wybor_uzytkownika):

"""
Function firstly makes list out of
user choice, then appends only those fcs which
are in gdb, then applies only domains which are wanted by user
(determined by fc choice)
"""
arcpy.env.workspace = work_space

lista_kolumn = wybor_uzytkownika.split(";")
all_tabele_gdb = arcpy.ListTables()
lista_poprawiona_o_kolumny =

for tabela in all_tabele_gdb:
pierwszy_czlon_nazwy = tabela.split("_")[0]
if pierwszy_czlon_nazwy in lista_kolumn:
lista_poprawiona_o_kolumny.append(tabela)
elif pierwszy_czlon_nazwy == 'man':
lista_poprawiona_o_kolumny.append(tabela)
else:
arcpy.Delete_management(tabela)

for tabela in lista_poprawiona_o_kolumny:
lista_robocza =
lista_robocza.append(tabela)
nazwa_domeny = lista_robocza[0]
arcpy.TableToDomain_management(
tabela, 'CODE', 'DESCRIPTION', work_space, nazwa_domeny, '-', 'REPLACE')
arcpy.Delete_management(tabela)

def selektor_pre(
baza_in, geometria, wojewodztwa,
kolumna, tymczasowa_nazwa, user_expression):

"""
Selects features based on user expression
"""
arcpy.env.workspace = baza_in

fc_lista = wojewodztwa.split(';')
arcpy.AddMessage(fc_lista)
for fc in fc_lista:
arcpy.MakeFeatureLayer_management(
fc, tymczasowa_nazwa)
arcpy.SelectLayerByAttribute_management(
tymczasowa_nazwa, "NEW_SELECTION", user_expression)
arcpy.CopyFeatures_management(
tymczasowa_nazwa, '{0}_{1}'.format(fc, kolumna))
arcpy.AddMessage(
'Seleckja skonczona dla {0}_{1}'.format(fc, kolumna))

def compare_save_to_csv(
gdb, pole_osm, xml_folder,
kolumna, parent,atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
csv_dir, nazwa_pliku, nazwa_id, nazwa_atrybutu,
user_geometry_choice):

"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')
if split[2] == kolumna and split[1] == user_geometry_choice:
czesc_nazwy = split[0]
geom = split[1]
nazwa_pliku = '{0}_{1}'.format(kolumna, geom)

lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
csv_dir, nazwa_id, nazwa_atrybutu,czesc_nazwy)

except IndexError:
pass

def compare_save_to_csv_wyjatek(
gdb, user_geometry_choice, user_column_choice,
pole_osm, xml_folder, kolumna, parent, atrybut_parent,
child, child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, csv_name, csv_name_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2):


"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser_wyjatki
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')

if split[2] == kolumna and split[1] == user_geometry_choice:

czesc_nazwy = split[0]
lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser_wyjatki(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent, atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, child_value_3, sciezka_csv,
csv_name, csv_name_2, nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy)

except IndexError:
pass

def merge_csv(
sciezka_csv, fragment_nazwy, nazwa_csv):
"""
Merges csv in specifed directory based on name scheme
"""
results = pd.DataFrame()
for counter, file in enumerate(glob.glob("{0}*{1}*".format(sciezka_csv, fragment_nazwy))):
name_dataframe = pd.read_csv(
file, usecols=[0, 1],encoding = 'CP1250' )
results = results.append(
name_dataframe)
results.to_csv(
'{0}{1}.csv'.format(sciezka_csv, nazwa_csv), encoding = 'CP1250')


def zapis_do_csv(
lista_1, lista_2, nazwa_1,
nazwa_2, csv_name, katalog,
czesc_nazwy):
"""
Saves to CSV, based on 2 lists.
"""
raw_data = {nazwa_1: lista_1,
nazwa_2: lista_2}
df = pd.DataFrame(raw_data, columns=[nazwa_1, nazwa_2])
df.to_csv(
'{0}{1}_{2}.csv'.format(katalog, czesc_nazwy, csv_name),
index=False, header=True, encoding = 'CP1250')


def xml_parser(
xml, lista_agis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
sciezka_csv, nazwa_id, nazwa_atrybutu,
czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))

zapis_do_csv(
l1, l2, nazwa_id,
nazwa_atrybutu, nazwa_pliku,
sciezka_csv, czesc_nazwy)

arcpy.AddMessage('Zapisalem {0}'.format(nazwa_pliku))
arcpy.AddMessage('Zapsialem tyle id: {0}'.format((len(l1))))
arcpy.AddMessage('Zapsialem tyle nazw: {0}'.format((len(l2))))

def xml_parser_wyjatki(
xml, lista_agis,
parent, atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, nazwa_pliku, nazwa_pliku_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

Added 'elif' for some feature classes that are described
by 2 value tags.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
l3 =
l4 =

for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))
elif sselement.attrib[child_atrybut] == child_value_3:
l3.append(
subelement.get(atrybut_parent))
l4.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))

zapis_do_csv(
l1, l2,
nazwa_id, nazwa_atrybutu,
nazwa_pliku, sciezka_csv, czesc_nazwy)

zapis_do_csv(
l3, l4,
nazwa_id, nazwa_atrybutu_2,
nazwa_pliku_2, sciezka_csv, czesc_nazwy)

def replace_csv(
csv, symbol_1, symbol_2):
'''
Function replace certain symbol to prevent
ArcGIS Pro from crashing during table import.
'''
my_csv_path = csv

with open(my_csv_path, 'r') as f:
my_csv_text = f.read()

find_str = symbol_1
replace_str = symbol_2

csv_str = re.sub(find_str, replace_str, my_csv_text)

with open(my_csv_path, 'w') as f:
f.write(csv_str)


def fix_field(
tabela , nazwa, pole):
"""
Imported tables has got not
valid field with ID. This fix that problem
by creating new on in text type, copying value
and deleting old one.
"""
arcpy.AddField_management(
tabela, nazwa, "TEXT", field_length = 20)
try:
with arcpy.da.UpdateCursor(tabela, [pole,nazwa]) as cursor:
for row in cursor:
row[1] = row[0]
cursor.updateRow(row)
except RuntimeError:
print(row[1])
del row,cursor
arcpy.DeleteField_management(tabela, [pole])


def import_fix_join(
in_table, out_gdb, nazwa,
id_csv, in_fc, field_osm,
pole_to_join):
"""
Imports table to geodatabase
Fixes its column
Join field to feature class.

"""
arcpy.TableToGeodatabase_conversion(
[in_table], out_gdb)

fix_field(
in_table, nazwa, id_csv)

pole = [pole_to_join]

arcpy.env.workspace = out_gdb

arcpy.JoinField_management(
in_fc, field_osm, in_table,
nazwa, pole)


Three scripts in ArcGIS Pro software.



enter image description here



Script number one GUI



enter image description here



Script number two GUI



enter image description here



Script number three GUI



enter image description here










share|improve this question















Background



My project is a Python toolbox. This is my first bigger coding project and it took me quite some time to code this toolbox. I learned a lot from the start point to this day. I changed this code hundreds of times, whenever I learned about new trick or method. This code works. I tested it with various datasets, fixed bugs or any syntax errors.



Cons:




  • The code ha some Polish names for: variable, functions etc.

  • GUI is all Polish

  • I started using Python about 3 months ago


What my code does:



Main purpose of this toolbox was to automate OpenStreetMap (OSM) data transformation from voivodeship shapefiles into country sized one, from which values were selected by their attributes to visualize features (for example, roads were selected and symbolized).



The code consists of three classes which are three scripts inside of my toolbox.



It is used in ArcGIS Pro to help non-programmer user to replicate my work.



My goal



Can someone who is more experienced than me in Python give me some useful advice?



Terms used in this code




  • shp - shapefile

  • osm - OpenStreetMap

  • fc - feature class

  • gdb - geodatabase


I added comments to my code to help understand what is happening.



My code



# -*- coding: CP1250 -*-
import arcpy
import os
import pandas as pd
import shutil
import xlrd
from xml.etree import ElementTree as ET
import glob
from itertools import starmap
import re


class Toolbox(object):
def __init__(self):
"""Define the toolbox (the name of the toolbox is the name of the
.pyt file)."""
self.label = "NardzedziaDoEskportu"
self.alias = ""

# List of tool classes associated with this toolbox
self.tools = [Przygotowanie_do_eksportu, SkryptDoEksportu, XML_export]


class SkryptDoEksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "OSM Polska"
self.description = "Skrypt eksportuje wybrane kolumny zawarte w tabeli atrybutow klas obiektow z geobazy."
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

# drugi parametr
klasy = arcpy.Parameter(
displayName="Warstwy w geobazie (mozliwy tylko podglad)",
name="fcs_of_gdb",
datatype="DEFeatureClass",
parameterType="Required",
direction="Input",
multiValue=True)

# trzeci parametr
kolumny = arcpy.Parameter(
displayName="Wybierz kolumny do selekcji",
name="colli",
datatype="GPString",
parameterType="Required",
direction="Input",
multiValue=True)
kolumny.filter.type = "ValueList"

# Czwarty parametr
plikExcel = arcpy.Parameter(
displayName="Plik *.XLS z domenami",
name="excelik",
datatype="DEType",
parameterType="Required",
direction="Input")

# Piaty parametr
plikShpWoj = arcpy.Parameter(
displayName="Plik *.Shp okreslajacy granice wojewodztw",
name="ShpWoj",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

# Szosty parametr
plikBoundary = arcpy.Parameter(
displayName="Plik *.Shp bedacy poprawiona wersja Polska_boundary_ply",
name="shpBoundary",
datatype="DEShapefile",
parameterType="Required",
direction="Input")

p = [inside, klasy, kolumny, plikExcel, plikShpWoj, plikBoundary]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""
parameters[1].enabled = 0

if parameters[0].value:
arcpy.env.workspace = parameters[0].value
fclist = arcpy.ListFeatureClasses()
parameters[1].value = fclist

if parameters[1].value:
fcs = parameters[1].value.exportToString()
single = fcs.split(";")
fields = arcpy.ListFields(single[0])
l1 = [f.name for f in fields]
l2 = ["OBJECTID", "Shape", "OSMID", "osmTags", "osmuser", "osmuid", "osmvisible",
"osmversion", "osmchangeset", "osmtimestamp", "osmMemberOf", "osmSupportingElement",
"osmMembers", " Shape_Length", "Shape_Area", "wayRefCount"]
l3 = [czynnik for czynnik in l1 if czynnik not in l2]
parameters[2].filter.list = l3
return

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""


return

class XML_export(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eksport danych z XML"
self.description = "Skrypt przygotowuje dane i eksportuje wybrane aspkety z XML"
self.canRunInBackground = False

def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr

inside = arcpy.Parameter(
displayName = "Wejsciowa geobaza",
name = "in_gdb",
datatype = "DEWorkspace",
parameterType = "Required",
direction = "Input",
multiValue = False)

# drugi parametr

rodzaj = arcpy.Parameter(
displayName = "Wybierz typ geometrii",
name = "geom",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)

rodzaj.filter.type = "ValueList"
rodzaj.filter.list = ['pt','ln','ply']

# trzeci parametr
klasy = arcpy.Parameter(
displayName = "Wybrane klasy",
name = "fcs_of_gdb",
datatype = "DEFeatureClass",
parameterType = "Required",
direction = "Input",
multiValue = True)

# czwarty
wojewodztwa_string = arcpy.Parameter(
displayName = "Wybierz wojewodztwa",
name = "colli",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = True)

wojewodztwa_string.filter.type = "ValueList"


#piaty
warstwa = arcpy.Parameter(
displayName = "Wybierz warstwe",
name = "fl_gdb",
datatype = "GPFeatureLayer",
parameterType = "Required",
direction = "Input")

# szosty
wyrazenie = arcpy.Parameter(
displayName = "Wpisz wyrazenie do selekcji",
name = "expres",
datatype = "GPSQLExpression",
parameterType = "Required",
direction = "Input")

wyrazenie.parameterDependencies = [warstwa.name]

# siodmy
folder_xml = arcpy.Parameter(
displayName = "Wskaz folder gdzie znajduja sie pliki w formacie XML",
name = "XMLdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

# osmy

folder_csv = arcpy.Parameter(
displayName = "Wskaz folder gdzie maja zostac zapisane pliki CSV",
name = "CSVdir",
datatype = "DEFolder",
parameterType = "Required",
direction = "Input")

#dziewiaty
kolumny = arcpy.Parameter(
displayName = "Wybierz kolumne",
name = "colli2",
datatype = "GPString",
parameterType = "Required",
direction = "Input",
multiValue = False)
kolumny.filter.type = "ValueList"

#dziesiaty
check_1 = arcpy.Parameter(
displayName = "Zaznacz aby dokonac zapisu do CSV (niezalecane odznaczanie)",
name = "check1",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)

check_1.value = True
#jedenasty
check_2 = arcpy.Parameter(
displayName = "Zaznacz aby polaczyc pliki CSV w jeden - odznaczenie spowoduje brak laczenia",
name = "check2",
datatype = "GPBoolean",
parameterType = "Optional",
direction = "Input",
multiValue = False)



p = [inside, rodzaj, klasy, wojewodztwa_string,
kolumny, warstwa, wyrazenie, folder_xml, folder_csv,
check_1, check_2]


return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""


wejsciowa_gdb = parameters[0]
wybrana_geometria = parameters[1]
lista_klas = parameters[2]
wybor_wojewodztwa = parameters[3]
wybor_kolumny = parameters[4]
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
lista_klas.enabled = 0

arcpy.env.workspace = wejsciowa_gdb.value
fclist = arcpy.ListFeatureClasses()
fc_o_wybranej_geometrii =

wybor = wybrana_geometria.valueAsText

if check_box_wartosc_2 and check_box_wartosc_1 == False:
parameters[0].enabled = 0
parameters[1].enabled = 0
parameters[3].enabled = 0
parameters[4].enabled = 0
parameters[5].enabled = 0
parameters[6].enabled = 0

if check_box_wartosc_1 and check_box_wartosc_2 == False:
parameters[0].enabled = 1
parameters[1].enabled = 1
parameters[3].enabled = 1
parameters[4].enabled = 1
parameters[5].enabled = 1
parameters[6].enabled = 1

for fc in fclist:
try:
split_nazwy = fc.split('_')
if len (split_nazwy) == 2 and split_nazwy[1] == wybor:
fc_o_wybranej_geometrii.append(fc)

except IndexError:
pass


lista_klas.value = fc_o_wybranej_geometrii

if lista_klas.value:
fcs = lista_klas.value.exportToString()
fcs_lista = fcs.split(";")
wybor_wojewodztwa.filter.list = fcs_lista

if wybrana_geometria.value:
if wybor == 'ln':
lista_ln = [
'highway', 'waterway', 'boundary'
]
wybor_kolumny.filter.list = lista_ln

elif wybor == 'pt':
lista_pt = [
'natural', 'aeroway', 'historic',
'leisure', 'waterway', 'shop',
'railway', 'tourism', 'highway',
'amenity'
]
wybor_kolumny.filter.list = lista_pt

elif wybor == 'ply':
lista_ply = [
'landuse', 'building', 'natural',
'amenity'
]
wybor_kolumny.filter.list = lista_ply



def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):
# Zmienne
# -*- coding: CP1250 -*-
arcpy.env.overwriteOutput = True
tymczasowa_nazwa = "tymczasowaNazwaDlaFC"
gdb = parameters[0].valueAsText
user_geometry_choice = parameters[1].valueAsText
user_wojewodztwo_choice = parameters[3].valueAsText
user_column_choice = parameters[4].valueAsText
user_expression = parameters[6].valueAsText
dir_xml = parameters[7].valueAsText
dir_csv = parameters[8].valueAsText
field_osm = 'OSMID'
xml_parent_way = 'way'
xml_parent_node = 'node'
xml_atr_parent = 'id'
xml_child = 'tag'
xml_atr_child = 'k'
xml_value_child_1 = 'name'
xml_value_child_2 = 'v'
xml_value_child_3 = 'ele'
xml_value_child_4 = 'addr:housenumber'
xml_value_child_5 = 'ref'
id_csv = 'id_robocze'
id_csv_2 = 'id_elementu'
nazwa_csv = 'nazwa'
natural_name = "nazwa_ele"
natural_name_2 = "wysokosc"
building_name = "budynki_nazwa"
building_name_2 = "buydnki_numery"
natural_csv_name = 'natural_nazwa'
natural_csv_name_2 = 'natural_wysokosc'
building_csv_name = 'budynki_nazwa'
building_csv_name_2 = 'budynki_numery'
highway_name = 'ulice'
highway_name_2 = 'nr_drogi'
highway_csv_name = 'ulice'
highway_csv_name_2 = 'nr_drogi'
check_box_wartosc_1 = parameters[9].value
check_box_wartosc_2 = parameters[10].value
dir_natural = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name,
user_geometry_choice))
dir_natural_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(natural_csv_name_2,
user_geometry_choice))
dir_any = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(user_column_choice,
user_geometry_choice))
dir_building = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name,
user_geometry_choice))
dir_building_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(building_csv_name_2,
user_geometry_choice))
dir_highway = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name,
user_geometry_choice))
dir_highway_2 = os.path.join(
dir_csv,'Polska_{0}_{1}.csv'.format(highway_csv_name_2,
user_geometry_choice))

# Selekcja z geobazy plikow, ktore zostana wykorzystane do stworzenia list fc
if check_box_wartosc_1:
selektor_pre(
gdb, user_geometry_choice, user_wojewodztwo_choice,
user_column_choice, tymczasowa_nazwa, user_expression)

get_csv(
gdb, user_geometry_choice, user_column_choice, field_osm, dir_xml,
xml_parent_node, xml_atr_parent, xml_child, xml_atr_child,
xml_value_child_1, xml_value_child_3, dir_csv, natural_csv_name,
natural_csv_name_2, id_csv, natural_name, natural_name_2,
xml_value_child_4, building_csv_name, building_csv_name_2,
building_name, building_name_2, xml_value_child_2, nazwa_csv,
xml_parent_way, highway_csv_name, highway_csv_name_2,
highway_name, highway_name_2, xml_value_child_5,
user_geometry_choice, user_column_choice, check_box_wartosc_1,
check_box_wartosc_2, id_csv_2, dir_natural, dir_natural_2, dir_any,
dir_building, dir_building_2, dir_highway, dir_highway_2)

return

class Przygotowanie_do_eksportu(object):
def __init__(self):
"""Define the tool (tool name is the name of the class)."""
self.label = "Eliminacja datasetow"
self.description = "Skrypt przygotowuje dane w geobazie, aby spelnialy wymagania nastepnego skryptu."
self.canRunInBackground = False


def getParameterInfo(self):
"""Define parameter definitions"""
# Pierwszy parametr
inside = arcpy.Parameter(
displayName="Wejsciowa geobaza",
name="in_gdb",
datatype="DEWorkspace",
parameterType="Required",
direction="Input")

p =[inside]
return p

def isLicensed(self):
"""Set whether tool is licensed to execute."""
return True

def updateParameters(self, parameters):
"""Modify the values and properties of parameters before internal
validation is performed. This method is called whenever a parameter
has been changed."""

def updateMessages(self, parameters):
"""Modify the messages created by internal validation for each tool
parameter. This method is called after internal validation."""

def execute(self, parameters, messages):

arcpy.env.overwriteOutput = True


arcpy.env.workspace = parameters[0].valueAsText
alt = arcpy.env.workspace
datalist = arcpy.ListDatasets()

#clears gdb out of data sets
for data in datalist:
for fc in arcpy.ListFeatureClasses("*", "ALL", data):
czesc = fc.split("_")
arcpy.FeatureClassToFeatureClass_conversion(
fc, alt, '{0}_{1}'.format(czesc[0], czesc[2]))
arcpy.Delete_management(data)


return

def import_excel(
in_excel, out_gdb):

"""
Opens excel file from path
Make a list from sheets in file
Iterates through sheets
"""
workbook = xlrd.open_workbook(in_excel)
sheets = [sheet.name for sheet in workbook.sheets()]
for sheet in sheets:
out_table = os.path.join(
out_gdb,
arcpy.ValidateTableName(
"{0}".format(sheet),
out_gdb))

arcpy.ExcelToTable_conversion(in_excel, out_table, sheet)

def iter_kolumny(
user_input, tymczasowa_mazwa,
warunek):
"""
Selection based on user choice

"""
lista_kolumn = user_input.split(";")
arcpy.AddMessage(
"Wybrales nastepujace parametry: {0}".format(lista_kolumn))
fc_lista = arcpy.ListFeatureClasses()
for fc in fc_lista:
czlon_nazwy = fc.split("_")
for kolumna in lista_kolumn:
arcpy.MakeFeatureLayer_management(fc, tymczasowa_mazwa)
try:
arcpy.SelectLayerByAttribute_management(
tymczasowa_mazwa, "NEW_SELECTION", '{0}{1}'.format(kolumna, warunek))
arcpy.CopyFeatures_management(
tymczasowa_mazwa, '{0}_{1}_{2}'.format(czlon_nazwy[0], kolumna, czlon_nazwy[1]))
except arcpy.ExecuteError:
pass
arcpy.Delete_management(fc)


def kolumny_split(
user_input, tymczasowa_mazwa, warunek,
gdb, wojewodztwa_shp, boundary_ply):
"""
After iter_kolumny call faulty column is deleted,
and new fc is imported which will be substitute for it

"""
iter_kolumny(
user_input, tymczasowa_mazwa, warunek)
arcpy.Delete_management(
'Polska_boundary_ply')
arcpy.FeatureClassToFeatureClass_conversion(
wojewodztwa_shp, gdb, 'GraniceWojewodztw')
arcpy.FeatureClassToFeatureClass_conversion(
boundary_ply, gdb, 'Polska_boundary_ply')

def listy_append(
listaFc, liniowa, polygon, punkty):
"""
Simple list appender

"""
for fc in listaFc:
czlon_nazwy = fc.split("_")
if czlon_nazwy[1] == "ln":
liniowa.append(fc)
elif czlon_nazwy[1] == "ply":
polygon.append(fc)
elif czlon_nazwy[1] == "pt":
punkty.append(fc)

def nadaj_domene(
work_space, wybor_uzytkownika):

"""
Function firstly makes list out of
user choice, then appends only those fcs which
are in gdb, then applies only domains which are wanted by user
(determined by fc choice)
"""
arcpy.env.workspace = work_space

lista_kolumn = wybor_uzytkownika.split(";")
all_tabele_gdb = arcpy.ListTables()
lista_poprawiona_o_kolumny =

for tabela in all_tabele_gdb:
pierwszy_czlon_nazwy = tabela.split("_")[0]
if pierwszy_czlon_nazwy in lista_kolumn:
lista_poprawiona_o_kolumny.append(tabela)
elif pierwszy_czlon_nazwy == 'man':
lista_poprawiona_o_kolumny.append(tabela)
else:
arcpy.Delete_management(tabela)

for tabela in lista_poprawiona_o_kolumny:
lista_robocza =
lista_robocza.append(tabela)
nazwa_domeny = lista_robocza[0]
arcpy.TableToDomain_management(
tabela, 'CODE', 'DESCRIPTION', work_space, nazwa_domeny, '-', 'REPLACE')
arcpy.Delete_management(tabela)

def selektor_pre(
baza_in, geometria, wojewodztwa,
kolumna, tymczasowa_nazwa, user_expression):

"""
Selects features based on user expression
"""
arcpy.env.workspace = baza_in

fc_lista = wojewodztwa.split(';')
arcpy.AddMessage(fc_lista)
for fc in fc_lista:
arcpy.MakeFeatureLayer_management(
fc, tymczasowa_nazwa)
arcpy.SelectLayerByAttribute_management(
tymczasowa_nazwa, "NEW_SELECTION", user_expression)
arcpy.CopyFeatures_management(
tymczasowa_nazwa, '{0}_{1}'.format(fc, kolumna))
arcpy.AddMessage(
'Seleckja skonczona dla {0}_{1}'.format(fc, kolumna))

def compare_save_to_csv(
gdb, pole_osm, xml_folder,
kolumna, parent,atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
csv_dir, nazwa_pliku, nazwa_id, nazwa_atrybutu,
user_geometry_choice):

"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')
if split[2] == kolumna and split[1] == user_geometry_choice:
czesc_nazwy = split[0]
geom = split[1]
nazwa_pliku = '{0}_{1}'.format(kolumna, geom)

lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
csv_dir, nazwa_id, nazwa_atrybutu,czesc_nazwy)

except IndexError:
pass

def compare_save_to_csv_wyjatek(
gdb, user_geometry_choice, user_column_choice,
pole_osm, xml_folder, kolumna, parent, atrybut_parent,
child, child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, csv_name, csv_name_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2):


"""
Iterates over feature classes in geodatabase,
checks for only those which user needs,
creates list of ids which will be used in xml_parser_wyjatki
"""
arcpy.env.workspace = gdb
wszystkie_fc = arcpy.ListFeatureClasses()

for fc in wszystkie_fc:
try:
split = fc.split('_')

if split[2] == kolumna and split[1] == user_geometry_choice:

czesc_nazwy = split[0]
lista_id_arcgis = [row[0]
for row in arcpy.da.SearchCursor(fc, pole_osm)]

arcpy.AddMessage("Dlugosc listy: {0}".format(
str(len(lista_id_arcgis))))

xml_parser_wyjatki(
'{0}{1}.xml'.format(xml_folder, czesc_nazwy),
lista_id_arcgis, parent, atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, child_value_3, sciezka_csv,
csv_name, csv_name_2, nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy)

except IndexError:
pass

def merge_csv(
sciezka_csv, fragment_nazwy, nazwa_csv):
"""
Merges csv in specifed directory based on name scheme
"""
results = pd.DataFrame()
for counter, file in enumerate(glob.glob("{0}*{1}*".format(sciezka_csv, fragment_nazwy))):
name_dataframe = pd.read_csv(
file, usecols=[0, 1],encoding = 'CP1250' )
results = results.append(
name_dataframe)
results.to_csv(
'{0}{1}.csv'.format(sciezka_csv, nazwa_csv), encoding = 'CP1250')


def zapis_do_csv(
lista_1, lista_2, nazwa_1,
nazwa_2, csv_name, katalog,
czesc_nazwy):
"""
Saves to CSV, based on 2 lists.
"""
raw_data = {nazwa_1: lista_1,
nazwa_2: lista_2}
df = pd.DataFrame(raw_data, columns=[nazwa_1, nazwa_2])
df.to_csv(
'{0}{1}_{2}.csv'.format(katalog, czesc_nazwy, csv_name),
index=False, header=True, encoding = 'CP1250')


def xml_parser(
xml, lista_agis, parent,
atrybut_parent, child, child_atrybut,
child_value_1, child_value_2, nazwa_pliku,
sciezka_csv, nazwa_id, nazwa_atrybutu,
czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))

zapis_do_csv(
l1, l2, nazwa_id,
nazwa_atrybutu, nazwa_pliku,
sciezka_csv, czesc_nazwy)

arcpy.AddMessage('Zapisalem {0}'.format(nazwa_pliku))
arcpy.AddMessage('Zapsialem tyle id: {0}'.format((len(l1))))
arcpy.AddMessage('Zapsialem tyle nazw: {0}'.format((len(l2))))

def xml_parser_wyjatki(
xml, lista_agis,
parent, atrybut_parent, child,
child_atrybut, child_value_1, child_value_2,
child_value_3, sciezka_csv, nazwa_pliku, nazwa_pliku_2,
nazwa_id, nazwa_atrybutu, nazwa_atrybutu_2, czesc_nazwy):
"""
Function to pick from xml files tag values.
Firstly it creates tree of xml file and then
goes each level down and when final condtion is fullfiled
id and value from xml file is appended to list in the end of
xml file list is saved to CSV.

Added 'elif' for some feature classes that are described
by 2 value tags.

"""
rootElement = ET.parse(xml).getroot()
l1 =
l2 =
l3 =
l4 =

for subelement in rootElement:
if subelement.tag == parent:
if subelement.get(atrybut_parent) in lista_agis:
for sselement in subelement:
if sselement.tag == child:
if sselement.attrib[child_atrybut] == child_value_1:
l1.append(
subelement.get(atrybut_parent))
l2.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))
elif sselement.attrib[child_atrybut] == child_value_3:
l3.append(
subelement.get(atrybut_parent))
l4.append(
sselement.get(child_value_2))
arcpy.AddMessage('Dodalem {0}'.format(sselement.get(child_value_2)))

zapis_do_csv(
l1, l2,
nazwa_id, nazwa_atrybutu,
nazwa_pliku, sciezka_csv, czesc_nazwy)

zapis_do_csv(
l3, l4,
nazwa_id, nazwa_atrybutu_2,
nazwa_pliku_2, sciezka_csv, czesc_nazwy)

def replace_csv(
csv, symbol_1, symbol_2):
'''
Function replace certain symbol to prevent
ArcGIS Pro from crashing during table import.
'''
my_csv_path = csv

with open(my_csv_path, 'r') as f:
my_csv_text = f.read()

find_str = symbol_1
replace_str = symbol_2

csv_str = re.sub(find_str, replace_str, my_csv_text)

with open(my_csv_path, 'w') as f:
f.write(csv_str)


def fix_field(
tabela , nazwa, pole):
"""
Imported tables has got not
valid field with ID. This fix that problem
by creating new on in text type, copying value
and deleting old one.
"""
arcpy.AddField_management(
tabela, nazwa, "TEXT", field_length = 20)
try:
with arcpy.da.UpdateCursor(tabela, [pole,nazwa]) as cursor:
for row in cursor:
row[1] = row[0]
cursor.updateRow(row)
except RuntimeError:
print(row[1])
del row,cursor
arcpy.DeleteField_management(tabela, [pole])


def import_fix_join(
in_table, out_gdb, nazwa,
id_csv, in_fc, field_osm,
pole_to_join):
"""
Imports table to geodatabase
Fixes its column
Join field to feature class.

"""
arcpy.TableToGeodatabase_conversion(
[in_table], out_gdb)

fix_field(
in_table, nazwa, id_csv)

pole = [pole_to_join]

arcpy.env.workspace = out_gdb

arcpy.JoinField_management(
in_fc, field_osm, in_table,
nazwa, pole)


Three scripts in ArcGIS Pro software.



enter image description here



Script number one GUI



enter image description here



Script number two GUI



enter image description here



Script number three GUI



enter image description here







python plugin geospatial arcpy






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited yesterday

























asked Nov 20 '18 at 15:59









JuniorPythonNewbie

1554




1554












  • Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
    – Reinderien
    Nov 20 '18 at 16:19


















  • Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
    – Reinderien
    Nov 20 '18 at 16:19
















Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
– Reinderien
Nov 20 '18 at 16:19




Writing the GUI to have Polish text is not a con if your users are Polish :) Internationalization is a huge and complex subject, but simply targeting one language is not necessarily a bad thing.
– Reinderien
Nov 20 '18 at 16:19










1 Answer
1






active

oldest

votes


















3














Some minor stuff:




  • I don't see where self.tools is used after initialization - can it be deleted? If you need to keep it, does it need to change? If it doesn't change (if it can be immutable), use a tuple instead of a list.

  • CP1250 should be avoided unless you have a really good reason. Everyone should be on UTF-8. Using UTF-8 will allow you to add all of the proper character accents in your strings, which currently appear to be missing.

  • Python's naming convention is snake_case for variables and function names, and UpperCamelCase only for classes, so canRunInBackground would actually be can_run_in_background. Same for other names.

  • Avoid naming list variables l1, l2, etc. They should have a meaningful name according to what they actually store.


  • For short function calls such as



    import_excel(
    excel, gdb)


    there is no need to split it onto two lines. For calls with many arguments it's fine, but here it's more legible on one line.




  • This:



    wejsciowa_gdb = parameters[0]
    wybrana_geometria = parameters[1]
    lista_klas = parameters[2]
    wybor_wojewodztwa = parameters[3]
    wybor_kolumny = parameters[4]


    can be abbreviated to



    wejsciowa_gdb, wybrana_geometria, lista_klas, wybor_wojewodztwa, wybor_kolumny = parameters[:5]


    there are similar instances elsewhere in your code.




  • I suggest making a loop for your checkbox logic:



    if check_box_wartosc_1 != check_box_wartosc_2:
    enabled = int(check_box_wartosc_1)
    for i in (0, 1, 3, 4, 5, 6):
    parameters[i] = enabled


  • After your if wybor == 'ln', you have several temporary list assignments. You don't need the temporary variables - you can assign the lists directly to filter.list.


  • The argument list for get_csv is a little insane. You should make a class with members for those arguments.







share|improve this answer



















  • 1




    To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
    – hjpotter92
    Nov 20 '18 at 22:44










  • 1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07










  • 4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07











Your Answer





StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
});
});
}, "mathjax-editing");

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "196"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f208077%2fpython-toolbox-for-openstreetmap-data%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









3














Some minor stuff:




  • I don't see where self.tools is used after initialization - can it be deleted? If you need to keep it, does it need to change? If it doesn't change (if it can be immutable), use a tuple instead of a list.

  • CP1250 should be avoided unless you have a really good reason. Everyone should be on UTF-8. Using UTF-8 will allow you to add all of the proper character accents in your strings, which currently appear to be missing.

  • Python's naming convention is snake_case for variables and function names, and UpperCamelCase only for classes, so canRunInBackground would actually be can_run_in_background. Same for other names.

  • Avoid naming list variables l1, l2, etc. They should have a meaningful name according to what they actually store.


  • For short function calls such as



    import_excel(
    excel, gdb)


    there is no need to split it onto two lines. For calls with many arguments it's fine, but here it's more legible on one line.




  • This:



    wejsciowa_gdb = parameters[0]
    wybrana_geometria = parameters[1]
    lista_klas = parameters[2]
    wybor_wojewodztwa = parameters[3]
    wybor_kolumny = parameters[4]


    can be abbreviated to



    wejsciowa_gdb, wybrana_geometria, lista_klas, wybor_wojewodztwa, wybor_kolumny = parameters[:5]


    there are similar instances elsewhere in your code.




  • I suggest making a loop for your checkbox logic:



    if check_box_wartosc_1 != check_box_wartosc_2:
    enabled = int(check_box_wartosc_1)
    for i in (0, 1, 3, 4, 5, 6):
    parameters[i] = enabled


  • After your if wybor == 'ln', you have several temporary list assignments. You don't need the temporary variables - you can assign the lists directly to filter.list.


  • The argument list for get_csv is a little insane. You should make a class with members for those arguments.







share|improve this answer



















  • 1




    To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
    – hjpotter92
    Nov 20 '18 at 22:44










  • 1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07










  • 4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07
















3














Some minor stuff:




  • I don't see where self.tools is used after initialization - can it be deleted? If you need to keep it, does it need to change? If it doesn't change (if it can be immutable), use a tuple instead of a list.

  • CP1250 should be avoided unless you have a really good reason. Everyone should be on UTF-8. Using UTF-8 will allow you to add all of the proper character accents in your strings, which currently appear to be missing.

  • Python's naming convention is snake_case for variables and function names, and UpperCamelCase only for classes, so canRunInBackground would actually be can_run_in_background. Same for other names.

  • Avoid naming list variables l1, l2, etc. They should have a meaningful name according to what they actually store.


  • For short function calls such as



    import_excel(
    excel, gdb)


    there is no need to split it onto two lines. For calls with many arguments it's fine, but here it's more legible on one line.




  • This:



    wejsciowa_gdb = parameters[0]
    wybrana_geometria = parameters[1]
    lista_klas = parameters[2]
    wybor_wojewodztwa = parameters[3]
    wybor_kolumny = parameters[4]


    can be abbreviated to



    wejsciowa_gdb, wybrana_geometria, lista_klas, wybor_wojewodztwa, wybor_kolumny = parameters[:5]


    there are similar instances elsewhere in your code.




  • I suggest making a loop for your checkbox logic:



    if check_box_wartosc_1 != check_box_wartosc_2:
    enabled = int(check_box_wartosc_1)
    for i in (0, 1, 3, 4, 5, 6):
    parameters[i] = enabled


  • After your if wybor == 'ln', you have several temporary list assignments. You don't need the temporary variables - you can assign the lists directly to filter.list.


  • The argument list for get_csv is a little insane. You should make a class with members for those arguments.







share|improve this answer



















  • 1




    To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
    – hjpotter92
    Nov 20 '18 at 22:44










  • 1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07










  • 4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07














3












3








3






Some minor stuff:




  • I don't see where self.tools is used after initialization - can it be deleted? If you need to keep it, does it need to change? If it doesn't change (if it can be immutable), use a tuple instead of a list.

  • CP1250 should be avoided unless you have a really good reason. Everyone should be on UTF-8. Using UTF-8 will allow you to add all of the proper character accents in your strings, which currently appear to be missing.

  • Python's naming convention is snake_case for variables and function names, and UpperCamelCase only for classes, so canRunInBackground would actually be can_run_in_background. Same for other names.

  • Avoid naming list variables l1, l2, etc. They should have a meaningful name according to what they actually store.


  • For short function calls such as



    import_excel(
    excel, gdb)


    there is no need to split it onto two lines. For calls with many arguments it's fine, but here it's more legible on one line.




  • This:



    wejsciowa_gdb = parameters[0]
    wybrana_geometria = parameters[1]
    lista_klas = parameters[2]
    wybor_wojewodztwa = parameters[3]
    wybor_kolumny = parameters[4]


    can be abbreviated to



    wejsciowa_gdb, wybrana_geometria, lista_klas, wybor_wojewodztwa, wybor_kolumny = parameters[:5]


    there are similar instances elsewhere in your code.




  • I suggest making a loop for your checkbox logic:



    if check_box_wartosc_1 != check_box_wartosc_2:
    enabled = int(check_box_wartosc_1)
    for i in (0, 1, 3, 4, 5, 6):
    parameters[i] = enabled


  • After your if wybor == 'ln', you have several temporary list assignments. You don't need the temporary variables - you can assign the lists directly to filter.list.


  • The argument list for get_csv is a little insane. You should make a class with members for those arguments.







share|improve this answer














Some minor stuff:




  • I don't see where self.tools is used after initialization - can it be deleted? If you need to keep it, does it need to change? If it doesn't change (if it can be immutable), use a tuple instead of a list.

  • CP1250 should be avoided unless you have a really good reason. Everyone should be on UTF-8. Using UTF-8 will allow you to add all of the proper character accents in your strings, which currently appear to be missing.

  • Python's naming convention is snake_case for variables and function names, and UpperCamelCase only for classes, so canRunInBackground would actually be can_run_in_background. Same for other names.

  • Avoid naming list variables l1, l2, etc. They should have a meaningful name according to what they actually store.


  • For short function calls such as



    import_excel(
    excel, gdb)


    there is no need to split it onto two lines. For calls with many arguments it's fine, but here it's more legible on one line.




  • This:



    wejsciowa_gdb = parameters[0]
    wybrana_geometria = parameters[1]
    lista_klas = parameters[2]
    wybor_wojewodztwa = parameters[3]
    wybor_kolumny = parameters[4]


    can be abbreviated to



    wejsciowa_gdb, wybrana_geometria, lista_klas, wybor_wojewodztwa, wybor_kolumny = parameters[:5]


    there are similar instances elsewhere in your code.




  • I suggest making a loop for your checkbox logic:



    if check_box_wartosc_1 != check_box_wartosc_2:
    enabled = int(check_box_wartosc_1)
    for i in (0, 1, 3, 4, 5, 6):
    parameters[i] = enabled


  • After your if wybor == 'ln', you have several temporary list assignments. You don't need the temporary variables - you can assign the lists directly to filter.list.


  • The argument list for get_csv is a little insane. You should make a class with members for those arguments.








share|improve this answer














share|improve this answer



share|improve this answer








edited Nov 21 '18 at 13:57

























answered Nov 20 '18 at 16:35









Reinderien

3,833821




3,833821








  • 1




    To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
    – hjpotter92
    Nov 20 '18 at 22:44










  • 1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07










  • 4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07














  • 1




    To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
    – hjpotter92
    Nov 20 '18 at 22:44










  • 1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07










  • 4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
    – JuniorPythonNewbie
    Nov 21 '18 at 9:07








1




1




To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
– hjpotter92
Nov 20 '18 at 22:44




To avoid confusion, you should mention that the snake_case convention is for variables/functions, and CamelCase for classes.
– hjpotter92
Nov 20 '18 at 22:44












1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
– JuniorPythonNewbie
Nov 21 '18 at 9:07




1# self tools has to stay like that - it is used by my software to create Toolbox which you can see in picture nr 1. 2# Thank you for that advice. I will change that 3# I read pep8 - i know about snake convention, for some names I just forgot to change it (I had some massive changes to do and everyday I find some stuff like this :P) but for some I don't think I can change that, beacuse python toolbox is made with a template (all self.something are not made by me, I can delete some of them such as displayName but cant change them, beacuse toolbox will crush )
– JuniorPythonNewbie
Nov 21 '18 at 9:07












4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
– JuniorPythonNewbie
Nov 21 '18 at 9:07




4# Been aware of that, It's my everyday struggle to have proper names. Thanks for pointing that out. #5,6,7,8,9 - thanks! New things to me, I will look into that. I knew something was wrong with get_csv, but I was unable to find solution! Big thanks for all your effort!
– JuniorPythonNewbie
Nov 21 '18 at 9:07


















draft saved

draft discarded




















































Thanks for contributing an answer to Code Review Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


Use MathJax to format equations. MathJax reference.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f208077%2fpython-toolbox-for-openstreetmap-data%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

is 'sed' thread safe

How to make a Squid Proxy server?