Creating a topic model for editions of the Equity published after a provincial election.

Project

Collect the editions of the Shawville Equity that published the results of provincial elections and create a topic model using R and Mallet.

Hypothesis

Provincial election coverage would contain topics and themes, such as unemployment, economic development and politics surrounding official languages that would be useful for investigation.

Method

  1. An R program collected the editions to topic model into a single directory.
  2. The dates of provincial elections from 1883-2010 were collected from www.quebecpolitique.com. The election dates are in line 41 in the R program.
  3. Each edition that was published within seven days of a provincial election was copied to a separate directory. See function lines 7-27.
  4. The collection of editions was processed using a copy of a topic modeller in R. [1]
  5. Added a different stopwords file and after repeated processing updated the stopwords based on results with thc, par, tho, les, des, p.m, a.m, pour, hie, lew, que, qui, e.m, otc, sur, ths and tbe. [2]
  6. Analysed the results.
  7. Changed values in the program.

Results

The resulting topics and cluster dendrogram are published below. For the processing performed on the text files for the provincial election coverage, the program had an error at this line:

km <- kmeans(topic_df_dist, n.topics) 
Error in sample.int(m, k) :
cannot take a sample larger than the population when 'replace = FALSE

However, when I ran the same program on a larger dataset consisting of 563 text files from 1970-1980 the program ran error free.  I conclude that the small number of text files for the provincial election coverage, twenty eight of them, caused the program not to function.  Results of 1970-1980 are below as well.

 

Citations:


[1] Graham, Shawn, Ian Milligan, Scott Weingart. "Topic Modeling with R" The Historian's Macroscope - working title. Under contract with Imperial College Press. Open Draft Version, Autumn 2013, http://themacroscope.org   https://github.com/shawngraham/R/blob/master/topicmodel.R

[2] Tapor, 2007. Tapor Tools Prototype. [online] Available at: http://taporware.ualberta.ca/~taporware/cgi-bin/prototype/glasgowstoplist.txt [Accessed 10 August 2017].

Analysis of Provincial Election Coverage Results.

Repeated processing and additions to the stopwords list refined the results.  However, these results do not provide a compelling list of topics that spanned provincial election results coverage.

Cluster Dendrogram of Provincial Election Coverage Results

Topics in Provincial Election Coverage

1 time people life long stock
2 saturday sale church weekend bedroom
3 congress bearings marquis paper spring
4 hodgins november parents nov laid
5 good work come know city
6 april mrs service good birthday
7 door son led grand gladstone
8 ont daughter wilson brown july
9 states probably mon ore vote
10 july august boys joan district
11 shawville phone sunday quebec bay
12 december christmas nov draveurs spence
13 toronto farm night ville sir
14 tin french flour alcohol irish
15 mrs miss june church service
16 december moved resolved christmas says
17 says mass local photo anniversary
18 clarendon passed municipality election thomas
19 old said house just men
20 september votes truck auto park
21 lord sugar trouble shot ion
22 lake monday middlemiss information game
23 man feet let hand thought
24 canada make best water public
25 pontiac school meeting wednesday bristol
26 young cure connection road session
27 new shawville use friends order
28 medicine child blood half council
29 lie portage fort ted royal
30 ami hut lie says island
31 years day great place canadian
32 war wanted august weeks god
33 coun deep moved killed near
34 british following special milk death
35 cents river tariff jordan dead

Analysis of the Equity 1970-1980.

Gathering topics from a larger corpus of editions of the Equity from 1970 - 1980 provided a more meaningful topic model.  The topics listed below on the right were grouped according to similarity into holidays/celebrations, Ontario, municipal, provincial, federal/national and use of French in the topic.

Holidays/celebrations

The largest number of topics fall into the group of holidays and celebrations. These are events such as Christmas, Easter, other holidays, fairs, carnivals and birthdays:

2 september august fair sept bride
3 march february january game carnival
5 december christmas january dec holiday
6 april easter spring seed trophy
15 canada day county services information
16 birthday robert stewart music farmers
32 anniversary schneiders celebrate celebrated youth

Does the large proportion of topics relating to celebrations and holidays reflect a desire by the publishers of the Equity to publish positive news for its English speaking readers during a period of uncertainty and debate surrounding language and Quebec's sovereignty during 1970-1980? Or did this reflect a desire to publish content that readers of the Equity would enjoy, regardless of their political opinions at this time?

Renfrew, Ontario

Two topics are about Renfrew county, Ontario. No other topics mention other counties in Québec nor Québec municipalities outside of the Pontiac. This may reflect a stronger connection between the readers of the Equity to the nearby, largely English speaking Ontario county of Renfrew than to other nearby parts of the province of Québec.

18 car sales renfrew radio door
28 end renfrew high contact tlie

Governments

Two topics, 8 and 25, concern municipal government.  Topic 29 is in French and may concern the province of Québec. Topic 21 may concern the Québec, Canadian or French government.   Topic 7 is a national topic. While this is a small number of topics as a sample, it is derived from a large corpus of weekly newspapers published over eleven years. Given that half of the topics in English that were grouped under municipal, provincial and federal/national are municipal, it is reasonable to say that this supports the hypothesis that the Equity concentrated more on local news rather than what was happening in the whole province during this period.

Municipal, Provincial and Federal/National Topics

8 meeting council moved motion road

25 seconded municipality given resolved letter

29 dans plus est québec vous

21 quebec canadian come french government

7 news show room mary national

Topics in French

Three topics are in French reflecting French content present in the Equity during this period.  Topic 29 concerns the province of Québec or its capital city.  In this set of topics, there are none in French relating to municipal government or national issues.

26 septembre ligne sécurité revenu antique
29 dans plus est québec vous
35 une pas être votre tout

Topics from editions of the Shawville Equity 1970-1980

 1 june july august summer aug                     
 2 september august fair sept bride                
 3 march february january game carnival            
 4 won class mcdowell radio tender                 
 5 december christmas january dec holiday          
 6 april easter spring seed trophy                 
 7 news show room mary national                    
 8 meeting council moved motion road               
 9 mrs ottawa home family held                     
10 pontiac sunday new wednesday time               
11 young girls dale jack hahn                      
12 photo boys eva ann win                          
13 year good page best like                        
14 saturday equity sale farm centennial            
15 canada day county services information          
16 birthday robert stewart music farmers           
17 phone saturday sale equity farm                 
18 car sales renfrew radio door                    
19 equity large president public showville         
20 phone saturday sale farm house                  
21 quebec canadian come french government          
22 free available club maple pork                  
23 october november nov seat oct                   
24 shawville service school bay church             
25 seconded municipality given resolved letter     
26 septembre ligne sécurité revenu antique         
27 present members place following parents         
28 end renfrew high contact tlie                   
29 dans plus est québec vous                       
30 house room tickets win line                     
31 campbell’s lbs evenings vote credit             
32 anniversary schneiders celebrate celebrated youth
33 ond store radio ladies mass                     
34 todd ski meehan saunders cord                   
35 une pas être votre tout

Comments

Add comment

Fields marked by '*' are required.
Comments are moderated. If you choose to make this comment public, it will not be visible to others until it is approved by the owner.

Reply to:

Public
Private: This reply will only be visible to you and the author of the preceeding comment.