Hi Rod,

Shame you didn't make it to govhack a month or so ago - this could of
been a project!

For gathering information from websites I would recommend using Python
along with requests and beautifulsoup4. For scraping/bots twitter
there are several python libraries - twitter, tweepy, and
twitterfollowbot. I'm not sure about getting data from FB.

Cheers,

William.

On Tue, Aug 30, 2016 at 7:01 AM, Roderick Aldridge
<rod.aldridge1@gmail.com> wrote:
> Others are doing the HCC site which is much better for finding things.
> Yes, I found the minutes on your link and they have some of the information
> we need. However votes are only recorded of a councilor requests a division
> which they seem to avoid whenever possible. Otherwise it is just X moved and
> Y seconded [Motion] Carried (ID). There are notes of discussions that
> sometimes mention issues and occasionally the councilors who raise them, but
> not very often.
> So some automated analysis would certainly be welcome. But it won't be easy.
> How do you search in a way that finds all the things they didn't do - the
> opportunities missed - the transport report that considers roads in great
> detail but rail is never mentioned as part of the development of a
> sustainable transport network. Or that they received a report on the
> significance of the great advances in electric vehicles and just received
> the report.They didn't decide to do anything to promote them in any way.
> Still anything that at least finds a good proportion would be very welcome.
>
> Also I found that there were no links to minutes before 2016. I spoke to
> staff member Mali Ahipene who told me
>
> "the webpage was recently updated to include only those agendas and minutes
> for 2016.�� However, we are about to reinstate the webpage with all of the
> agenda���s for the current triennium. Check back in an hour, and the
> information should be available. If you still have questions, feel free to
> give me a call."
> When I checked at 5 o'clock and again at 9 o'clock and again this morning I
> found that the uppdate had not happened - or not succeeded:
>
> Although the website indicates that records of the minutes are available
> from 28 November-2013 I can't find links to any before Tuesday 2 February
> 2016. I can't yet find any links for these before 2016 except for resource
> hearings going back to November 2015, and Significant consent applications,
> hearings and decisions.
> Links to results for a search on "minutes 2013" give pages with 2016 minutes
> only for the first 4 pages of results. The 5th page and later results give
> 2016 links or an Incorrect Link message.
>
> I would also like tips for gathering relevant information from public
> sources on websites, social media and anywhere.
>
> Rod
>
> On 29 August 2016 at 19:58, Lawrence D'Oliveiro <ldo@geek-central.gen.nz>
> wrote:
>>
>> On Mon, 29 Aug 2016 18:33:54 +1200, Roderick Aldridge wrote:
>>
>> > It is proving difficult to extract information from
>> > the Waikato Regional Council website, though the Hamilton City
>> > Council one is much better.
>>
>> I���ve done some website scraping.
>>
>> I assume the info you want from Environment Waikato is things like
>> minutes of meetings
>>
>> <http://www.waikatoregion.govt.nz/Community/Whats-happening/Council-meetings/Agendas-and-minutes-for-council-and-standing-committees-from-28-November-2013/>?
>> I see downloadable PDF files there. Similarly with the HCC website
>>
>> <http://www.hamilton.govt.nz/our-council/meetings-and-minutes/Pages/default.aspx>.
>>
>> Do you want to do some automated analysis of these?
>> _______________________________________________
>> wlug mailing list | wlug@list.waikato.ac.nz
>> Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
>
>
>
> _______________________________________________
> wlug mailing list | wlug@list.waikato.ac.nz
> Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
>
_______________________________________________
wlug mailing list | wlug@list.waikato.ac.nz
Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug