# Cities for Citizens: Identification of Public Values and their Conflicts in Urban Spaces"

This is the accompanying code to the Master Thesis research project "Cities for Citizens: Identification of Public Values and their Conflicts in Urban Spaces" conducted in cooperation with the CUSP Lab @ TPM TU Delft and the City Science Lab Hamburg. Aiming to identify public values and their spatial conflicts, a total of nine Jupyter notebooks and one R script follow the research flow as outlined in the Master thesis document. Their content is described below in further detail. 

**Structure**: Basically all the operations and data mingling / modelling are done in the notebooks themselves. Some (recurring) methods have been outsourced in the ```valueconflicts.py``` scripts in order to bundle main functions in one package. 

> *Important Note*: Due to privacy regulations, no raw data of citizen contributions can be shared within this repository. However, given API access of Hamburg's digital participation system (DIPAS), the code is (re)usable to download and analyze future participatory projects. 

## Prerequesites
* Python 3.x
* Packages and dependencies in ```environment.yml```


## 1 - Data Collection
This notebook provides the functions to download the participatory data (project areas, geo-located contributions, comments, replies) from the DIPAS platform. In order for the script to function, one needs API access to DIPAS.

## 2 - Data Preparation
This notebook prepares the data for structural topic modelling (STM). It performs basic data cleaning operations, translates the participatory data from German to English and conducts the preprocessing operations needed for STM. 

## 3 - Sentiment Analysis
Leveraging the VADER library, the translated contributions, comments and replies are analyzed for their sentiments.

## 4 - Topic Modelling
Both the notebook and the R script named topic modelling contain ways to apply either latent dirichlet allocation (LDA, in Python) or structural topic modelling (STM, in R) to the participatory data

## 5 - Experiments
This notebook contains interactive Jupyter widgets to create and modify experiments that guide the exploratory research process. Experiments are read from and saved to ```data/02_experiments/experiments.csv```. The subsequent notebooks always refer back to the experimental framework to load and save data accordingly.
This notebook also contains multiple scripts that prepare the data in different ways for varying experiments.

## 6 - Value Assignment
This notebooks main content is an interactive Jupyter notebook that allows a user to interactively navigate through the resulting topics of the STM algorithm and assign both topic names and public values to them. The results are then saved and wordclouds for the respective value clusters can be created. 

## 7 - Spatial Value Clustering
Leveraging the HDBSCAN and the alpha shape algorithm, the public values collections are analyzed for spatial clusters. The resulting clusters are saved in an interactive folium map

## 8 - Spatial Topic Clustering
Similar purpose as notebook 7. Only difference is that now clusters of all topics, not only aggregated values are calculated.

## 9 - Visualizations
The name is the content; different visualizations for different (spatial) data.