Yes, you can and I do that.
I have a wi-fi router which I use in my house, and a LAN cable between our two houses which my neighbour shares. Since the router is always connected to the network, my neighbour can use her computer any time. And of course she pays a portion of the monthly bill.
It all depends on the configuration of your house or flat in relation to your neighbour. Obviously across the street is difficult, unless you use a wi-fi router and an aerial, but in my case, the cable goes between our two houses, a distance of about 12 metres between the houses and about 30m of cable altogether.