Tigor.nl

Tigor Sinuraja personal website

  • Increase font size
  • Default font size
  • Decrease font size
Home Sharing thoughts

Sharing thoughts

E-mail Print PDF

2022-01-02 An idea how the Cloud version of my application is going to be.

  • WebScraping of training data - Azure Function (Python)
  • Store collected data - Azure Cosmos DB
  • Data preparation - Databricks and Azure Data Factory
  • Training a model and generate inference model - Azure Automated ML as low code as possible( later may be Databricks and Azure ML SDK)
  • Getting instances to predict - Azure Function (Python)
  • Ad hoc analysis - Azure Synapse Serverless pool
  • Visualization - PowerBI
  • Versioning and deployment - Azure Devops

 

 

 

2022-01-01 A good day to begin with something new! So I am going to write down here whatever I think is worth remembering. In the free time between two jobs I did what I find more entertaining than watching Netflix or even playing chess. I am busy with converting my MMA classifier to Azure, The first step is to use serverless Azure functions for web-scraping instead of locally running the Python code. I use Visual Code for development, Azure Devops repo voor versioning and Azure pipeline for deployment. After quite a number of attempts, mainly struggling with Python dependencies install at deployment, its finally working. What a joy! Azure Function App scrapes Sherdog.com for UFC events and load the data into a contaner in CosmosDB. Why the heck there? Why not? I must say I find loading JSON into a CosmosDB container without worrying about defining the schema first somewhat liberating. Schema on Read is the way to go. Or may be I am just tired of thinking before I can load data in a database like SQL Server. "Just do it!".

 

Last Updated on Sunday, 02 January 2022 09:36  

Polls

Which tool is the best?
 

Statistieken

Members : 98
Content : 63
Web Links : 6
Content View Hits : 34197

Who's Online

We have 4 guests online

Interessante links

Featured Links:
Casemix
Advies- en onderzoeksbureau voor de zorgsector
NZa
de Nederlandse Zorgautoriteit
BI future blog
Blog van Hennie de Nooijer