Researching information on local body candidates
Hi everybody I am researching information on local body election candidates, concentrating on sustainable environmental and social policies. I am particularly interested in the track record of councillors who are standing again. It is proving difficult to extract information from the Waikato Regional Council website, though the Hamilton City Council one is much better. Would anybody like to share their expertise on ferreting information from websites, and for gathering relevant information from public sources on websites, social media and anywhere. Or do you know anybody who ma be willing to share their skills. Any help much appreciated. Rod
On Mon, 29 Aug 2016 18:33:54 +1200, Roderick Aldridge wrote:
It is proving difficult to extract information from the Waikato Regional Council website, though the Hamilton City Council one is much better.
I’ve done some website scraping. I assume the info you want from Environment Waikato is things like minutes of meetings <http://www.waikatoregion.govt.nz/Community/Whats-happening/Council-meetings/Agendas-and-minutes-for-council-and-standing-committees-from-28-November-2013/>? I see downloadable PDF files there. Similarly with the HCC website <http://www.hamilton.govt.nz/our-council/meetings-and-minutes/Pages/default.aspx>. Do you want to do some automated analysis of these?
Others are doing the HCC site which is much better for finding things. Yes, I found the minutes on your link and they have some of the information we need. However votes are only recorded of a councilor requests a division which they seem to avoid whenever possible. Otherwise it is just X moved and Y seconded [Motion] Carried (ID). There are notes of discussions that sometimes mention issues and occasionally the councilors who raise them, but not very often. So some automated analysis would certainly be welcome. But it won't be easy. How do you search in a way that finds all the things they didn't do - the opportunities missed - the transport report that considers roads in great detail but rail is never mentioned as part of the development of a sustainable transport network. Or that they received a report on the significance of the great advances in electric vehicles and just received the report.They didn't decide to do anything to promote them in any way. Still anything that at least finds a good proportion would be very welcome. Also I found that there were no links to minutes before 2016. I spoke to staff member Mali Ahipene who told me "the webpage was recently updated to include only those agendas and minutes for 2016. However, we are about to reinstate the webpage with all of the agenda’s for the current triennium. Check back in an hour, and the information should be available. If you still have questions, feel free to give me a call." When I checked at 5 o'clock and again at 9 o'clock and again this morning I found that the uppdate had not happened - or not succeeded: Although the website indicates that records of the minutes are available from 28 November-2013 I can't find links to any before Tuesday 2 February 2016. I can't yet find any links for these before 2016 except for resource hearings going back to November 2015, and Significant consent applications, hearings and decisions. Links to results for a search on "minutes 2013" give pages with 2016 minutes only for the first 4 pages of results. The 5th page and later results give 2016 links or an Incorrect Link message. I would also like tips for gathering relevant information from public sources on websites, social media and anywhere. Rod On 29 August 2016 at 19:58, Lawrence D'Oliveiro <ldo(a)geek-central.gen.nz> wrote:
On Mon, 29 Aug 2016 18:33:54 +1200, Roderick Aldridge wrote:
It is proving difficult to extract information from the Waikato Regional Council website, though the Hamilton City Council one is much better.
I’ve done some website scraping.
I assume the info you want from Environment Waikato is things like minutes of meetings <http://www.waikatoregion.govt.nz/Community/Whats- happening/Council-meetings/Agendas-and-minutes-for-council-and-standing- committees-from-28-November-2013/>? I see downloadable PDF files there. Similarly with the HCC website <http://www.hamilton.govt.nz/our-council/meetings-and- minutes/Pages/default.aspx>.
Do you want to do some automated analysis of these? _______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
Hi Rod, Shame you didn't make it to govhack a month or so ago - this could of been a project! For gathering information from websites I would recommend using Python along with requests and beautifulsoup4. For scraping/bots twitter there are several python libraries - twitter, tweepy, and twitterfollowbot. I'm not sure about getting data from FB. Cheers, William. On Tue, Aug 30, 2016 at 7:01 AM, Roderick Aldridge <rod.aldridge1(a)gmail.com> wrote:
Others are doing the HCC site which is much better for finding things. Yes, I found the minutes on your link and they have some of the information we need. However votes are only recorded of a councilor requests a division which they seem to avoid whenever possible. Otherwise it is just X moved and Y seconded [Motion] Carried (ID). There are notes of discussions that sometimes mention issues and occasionally the councilors who raise them, but not very often. So some automated analysis would certainly be welcome. But it won't be easy. How do you search in a way that finds all the things they didn't do - the opportunities missed - the transport report that considers roads in great detail but rail is never mentioned as part of the development of a sustainable transport network. Or that they received a report on the significance of the great advances in electric vehicles and just received the report.They didn't decide to do anything to promote them in any way. Still anything that at least finds a good proportion would be very welcome.
Also I found that there were no links to minutes before 2016. I spoke to staff member Mali Ahipene who told me
"the webpage was recently updated to include only those agendas and minutes for 2016. However, we are about to reinstate the webpage with all of the agenda’s for the current triennium. Check back in an hour, and the information should be available. If you still have questions, feel free to give me a call." When I checked at 5 o'clock and again at 9 o'clock and again this morning I found that the uppdate had not happened - or not succeeded:
Although the website indicates that records of the minutes are available from 28 November-2013 I can't find links to any before Tuesday 2 February 2016. I can't yet find any links for these before 2016 except for resource hearings going back to November 2015, and Significant consent applications, hearings and decisions. Links to results for a search on "minutes 2013" give pages with 2016 minutes only for the first 4 pages of results. The 5th page and later results give 2016 links or an Incorrect Link message.
I would also like tips for gathering relevant information from public sources on websites, social media and anywhere.
Rod
On 29 August 2016 at 19:58, Lawrence D'Oliveiro <ldo(a)geek-central.gen.nz> wrote:
On Mon, 29 Aug 2016 18:33:54 +1200, Roderick Aldridge wrote:
It is proving difficult to extract information from the Waikato Regional Council website, though the Hamilton City Council one is much better.
I’ve done some website scraping.
I assume the info you want from Environment Waikato is things like minutes of meetings
<http://www.waikatoregion.govt.nz/Community/Whats-happening/Council-meetings/Agendas-and-minutes-for-council-and-standing-committees-from-28-November-2013/>? I see downloadable PDF files there. Similarly with the HCC website
<http://www.hamilton.govt.nz/our-council/meetings-and-minutes/Pages/default.aspx>.
Do you want to do some automated analysis of these? _______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
_______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
Thanks Lawrence and William. Any chance either of you could give me a hands-on demo at a time and place to suit you? I am happy for you to find info for me, but I would like to learn how to do it myself. The criteria used to evaluate the candidates are in the Vision statement at http://sustainablewaikato.org.nz/vision.html Not easy to base a query on. Rod On 30 August 2016 at 07:18, William Mckee <will(a)artcontrol.me> wrote:
Hi Rod,
Shame you didn't make it to govhack a month or so ago - this could of been a project!
For gathering information from websites I would recommend using Python along with requests and beautifulsoup4. For scraping/bots twitter there are several python libraries - twitter, tweepy, and twitterfollowbot. I'm not sure about getting data from FB.
Cheers,
William.
Others are doing the HCC site which is much better for finding things. Yes, I found the minutes on your link and they have some of the information we need. However votes are only recorded of a councilor requests a
On Tue, Aug 30, 2016 at 7:01 AM, Roderick Aldridge <rod.aldridge1(a)gmail.com> wrote: division
which they seem to avoid whenever possible. Otherwise it is just X moved and Y seconded [Motion] Carried (ID). There are notes of discussions that sometimes mention issues and occasionally the councilors who raise them, but not very often. So some automated analysis would certainly be welcome. But it won't be easy. How do you search in a way that finds all the things they didn't do - the opportunities missed - the transport report that considers roads in great detail but rail is never mentioned as part of the development of a sustainable transport network. Or that they received a report on the significance of the great advances in electric vehicles and just received the report.They didn't decide to do anything to promote them in any way. Still anything that at least finds a good proportion would be very welcome.
Also I found that there were no links to minutes before 2016. I spoke to staff member Mali Ahipene who told me
"the webpage was recently updated to include only those agendas and minutes for 2016. However, we are about to reinstate the webpage with all of the agenda’s for the current triennium. Check back in an hour, and the information should be available. If you still have questions, feel free to give me a call." When I checked at 5 o'clock and again at 9 o'clock and again this morning I found that the uppdate had not happened - or not succeeded:
Although the website indicates that records of the minutes are available from 28 November-2013 I can't find links to any before Tuesday 2 February 2016. I can't yet find any links for these before 2016 except for resource hearings going back to November 2015, and Significant consent applications, hearings and decisions. Links to results for a search on "minutes 2013" give pages with 2016 minutes only for the first 4 pages of results. The 5th page and later results give 2016 links or an Incorrect Link message.
I would also like tips for gathering relevant information from public sources on websites, social media and anywhere.
Rod
On 29 August 2016 at 19:58, Lawrence D'Oliveiro <ldo(a)geek-central.gen.nz
wrote:
On Mon, 29 Aug 2016 18:33:54 +1200, Roderick Aldridge wrote:
It is proving difficult to extract information from the Waikato Regional Council website, though the Hamilton City Council one is much better.
I’ve done some website scraping.
I assume the info you want from Environment Waikato is things like minutes of meetings
happening/Council-meetings/Agendas-and-minutes-for-council-and-standing- committees-from-28-November-2013/>?
I see downloadable PDF files there. Similarly with the HCC website
<http://www.hamilton.govt.nz/our-council/meetings-and- minutes/Pages/default.aspx>.
Do you want to do some automated analysis of these? _______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
_______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
_______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
On Tue, 30 Aug 2016 07:54:45 +1200, Roderick Aldridge wrote:
The criteria used to evaluate the candidates are in the Vision statement at http://sustainablewaikato.org.nz/vision.html Not easy to base a query on.
Hmmm, this sounds more nebulous than what I had in mind. I thought there would be numbers that we could tally in some more manageable form (à la that PyCon video on sports stats that William showed us). This seems more like trying to gauge the tenor of councillors’ utterances to try to place them on some vague kind of spectrum. Unless you had some manual preprocessing in mind, to go through the records to assign scores to each councillor...
Thanks William, Lawrence, Ian, and Wayne I hope this is not too off topic for others and you want it to be taken it off list. Finding meaningful information among the plethora of stuff out there, particularly information on the actions of government bodies is important for all of us, and Open Source can play a key role in freedom of information. I found that agendas and minutes earlier than 2016 had been removed from the WRC website. They have undertaken to supply them on what they seem to be treating as an Official Information Request. What would help me most is ways of finding information on the Waikato Regional Council candidates - anything about their attitudes to looking after our environment and .ensuring we have a good future. I can supply a list of WRC candidates. Others are doing the Hamilton City council candidates. Any way of finding relevant info on the candidates - then searching it for key ideas, policies, track record of actions, particularly current councilors standing again I fear there is no substitute for reading the selected documents and judging their significance. So, finding info from official sources, websites, media, Facebook etc, etc. Searching the info for key words and phrases. Searching multiple documents particularly PDFs. Anything that helps select the useful from the rest. Any help welcome Let me know if you would like a list of WRC candiates Rod On 30 August 2016 at 11:38, Lawrence D'Oliveiro <ldo(a)geek-central.gen.nz> wrote:
On Tue, 30 Aug 2016 07:54:45 +1200, Roderick Aldridge wrote:
The criteria used to evaluate the candidates are in the Vision statement at http://sustainablewaikato.org.nz/vision.html Not easy to base a query on.
Hmmm, this sounds more nebulous than what I had in mind. I thought there would be numbers that we could tally in some more manageable form (à la that PyCon video on sports stats that William showed us). This seems more like trying to gauge the tenor of councillors’ utterances to try to place them on some vague kind of spectrum.
Unless you had some manual preprocessing in mind, to go through the records to assign scores to each councillor... _______________________________________________ wlug mailing list | wlug(a)list.waikato.ac.nz Unsubscribe: https://list.waikato.ac.nz/mailman/listinfo/wlug
Hi Rod,
I am particularly interested in the track record of councillors who are standing again.
A few years ago, when the hot topics were reducing HCC's $400M debt, V8 racing, and the Claudelands Event centre, there was a Waikato Times article that featured, then councillor, Ewan Wilson and I recollect that he provided statistics of the position each councillor had taken in their voting on various issues. Unfortunately I haven't had any luck in finding this article by using google. However I did come across the http://www.concernedcitizen.co.nz/ web-site, which has quite a lot of links to Waikato Times articles. Maybe these concerned citizen folks used to keep the statistics of the voting. If councillor voting statistics are not minuted and made public by the HCC on their web-site, then maybe you need to approach a councillor (or former councillor) in the hope that they have kept their own records of the voting. I assume that a Waikato Times reporter attends each council meeting, so the Waikato Times may also have a record of all the voting statistics. I also recollect that the Hamilton News used to have the "Robinson Report", and from reading these reports it appeared that they regularly attended HCC meetings. Maybe you could contact Geoffrey and Reihana Robinson and see if they kept voting statistics. robinsonsreport(a)gmail.com cheers, Ian.
participants (4)
-
Ian Stewart -
Lawrence D'Oliveiro -
Roderick Aldridge -
William Mckee