Malaysia Economic Outlook 2027 Analytics Suite docs

This repository contains 10 data science and data engineering projects built on the Ministry of Finance Malaysia Economic Outlook 2027 report (258 pages, released 9 October 2026).

Architecture Overview

The suite uses a single Python codebase with a relational SQLite database, clean CSV exports, Pydantic schemas, and a Bauhaus functionalist Streamlit workbench.

Module	Project Name	Source Tables / Pages	Core Function
`p01_state_cpi.py`	State-Level CPI Dashboard	Table 3.16 (pp. 212-213)	Computes a Retail Pressure Index (RPI) for Klang Valley, Pulau Pinang, Sabah, and Sarawak.
`p02_retail_forecaster.py`	HYGR Retail Demand Forecaster	Table 3.5, 3.13, 3.19	Links Private Consumption, CPI, and Employment to HYGR sales across Shopee MY, Shopee SG, TikTok MY, and TikTok SG.
`p03_etl_pipeline.py`	Report-to-Dataset Pipeline	Tables 1.1 to 5.1 (pp. 185-254)	Extracts 14 canonical appendix tables into CSV and SQLite with SHA-256 hash verification.
`p04_report_rag.py`	Chat with the Report (RAG)	Full 258-Page Report	Hybrid TF-IDF/n-gram and Local LLM (LM Studio / Ollama) Q&A tool with page citations.
`p05_gdp_nowcast.py`	GDP Nowcasting	Tables 3.1, 3.2, 3.3	Predicts GDP growth from the Index of Services and Industrial Production Index against the 4.2% to 5.2% target range.
`p06_trade_network.py`	Trade Partner Analysis	Tables 3.7, 3.10, 3.12 + Box 3.2	Sankey trade network, Herfindahl-Hirschman Index (HHI) concentration, GTRI 2026 score, and Strait of Hormuz exposure.
`p07_inflation_drivers.py`	Inflation Drivers	Tables 3.13 to 3.18 + Art. 1.2	Models PPI stage-of-processing pass-through to CPI and simulates Brent crude oil and fuel subsidy shocks.
`p08_labour_clustering.py`	Labour Market Clustering	Tables 3.19 to 3.22	K-Means clustering on employment growth, vacancies, and placements to isolate skills gap sectors.
`p09_banking_stress.py`	Banking & Financial Stress	Tables 4.1 to 4.6	Commercial and Islamic loan analysis, MGS vs corporate bond yield curves, and credit growth Z-score anomaly detection.
`p10_sdg_tracker.py`	SDG Progress Tracker	Table 5.1 (SDGs 1 to 17)	Scores Malaysia’s progress across all 17 SDGs and flags regressing indicators.

Getting Started

Copy the environment template:
```
cp .env.example .env
```
Install the Python dependencies:
```
pip install -r requirements.txt
```
Run the automated verification suite:
```
pytest -v
```
Launch the interactive workbench:
```
streamlit run app.py
```

Data Privacy

Raw HYGR SQL dumps and raw platform CSV exports are excluded via .gitignore. The repository includes only normalized index series (2025-Q1 = 100) across Shopee MY, Shopee SG, TikTok MY, and TikTok SG.