Street Address Comparisions -> How?

[Pho3nix]

Hi,

Need to do this at the office. Off the top of my head, character matching sounds like a good first step.
Looking also at using a synonym file for deviations of street, str, strt etc.
Also pondering this but not really understanding it

Thoughts and suggestions?

 

[SlinkyMike]

Soundex. Used it successfully to group different spellings of streets in the past.

 

[halfmoonforever]

I will book you a seat in the mental asylum after your attempt. I've been handling address information for a good 12 years, and I can tell you now, the same person can spell and type in their address countless amount of ways. Not to mention several others capturing the exact same information. Don't even get me started on postal code usage .... good luck... your booking is valid for 1 year

 

[Nemus]

I will book you a seat in the mental asylum after your attempt. I've been handling address information for a good 12 years, and I can tell you now, the same person can spell and type in their address countless amount of ways. Not to mention several others capturing the exact same information. Don't even get me started on postal code usage .... good luck... your booking is valid for 1 year

Also considering Main Road vs Main Street vs Mainstreet - Hoofstraat vs Hoof straat

 

[Pho3nix]

I will book you a seat in the mental asylum after your attempt. I've been handling address information for a good 12 years, and I can tell you now, the same person can spell and type in their address countless amount of ways. Not to mention several others capturing the exact same information. Don't even get me started on postal code usage .... good luck... your booking is valid for 1 year

Lol need to compare vs source. 70% acceptance rate else ignore so don't think I'll too insane.. hopefully :erm:

 

[Pho3nix]

Thoughts on libpostal on github?

 

[gkm]

Good luck. 70% is pretty high. I worked with addresses for a year and promised myself I am never going to take another job that involves them in any significant way.

 

[John_Phoenix]

Stock codes from multiple suppliers/data sources, matched to names and then reverse matching names to sku. No fun at all. How about scoring the strings? Eg. Hoof Straat 21120222112, vowels and consonants as an additional check. Or the long way around, generate variations off the bat and check against all of them. And when you find a pattern, add it to all.

 

[[)roi(]]

Thoughts on libpostal on github?

NLP is a good approach to this & Libpostal is a good option.
Alternatively build your own simple classification NLP off a probability algorithm like Naive Bayes classifier.

 

[XennoX]

NLP is a good approach to this & Libpostal is a good option.
Alternatively build your own simple classification NLP off a probability algorithm like Naive Bayes classifier.

This.

Guys at my office are doing just that.

 

[Pho3nix]

This.

Guys at my office are doing just that.

Wanna ask them for their code
Gonna start actual dev come next week.

 

[XennoX]

Wanna ask them for their code
Gonna start actual dev come next week.[/QUOTE

A map service with an API like Google Maps or OpenStreetMap.org may be your friend.

Lol. Code.

For NLP you need data, lots of it. Our one project that we did, we downloaded the entire English dictionary so we could build a suitable Word2Vec model. We settled on a, reasonable, 300 dimension vector. The data requirements to train such a model is mind boggling for to an individual. The model consumed 1 PB of data and cost my company in the region of US$14000 for the computation time and data consumption.

 

[Pho3nix]

Wanna ask them for their code
Gonna start actual dev come next week.[/QUOTE

A map service with an API like Google Maps or OpenStreetMap.org may be your friend.

Lol. Code.

For NLP you need data, lots of it. Our one project that we did, we downloaded the entire English dictionary so we could build a suitable Word2Vec model. We settled on a, reasonable, 300 dimension vector. The data requirements to train such a model is mind boggling for to an individual. The model consumed 1 PB of data and cost my company in the region of US$14000 for the computation time and data consumption.

Have the data but don't know if business can wait for the model to be trained :|
Will mention in tomorrow's stand-up.

Problem I'm also pondering is how much self tagging will be suitable for the model to have a good base

As a side note, I'm guessing this isn't the fastest machine ?

 

[_kabal_]

Thoughts on libpostal on github?

Wow that is awesome