Day 1 Responses from the Catalyst survey

Respondents were approached via the Catalyst mailing list (approximately 1000 subscribers), a post to many Perl Mongers mailing lists, and a post on the Perlbuzz weblog (approximately 1800 subscribers via Feedburner). 100 responses were received within 24 hours, and with no reminders, a total of 153 responses were received in one week. There were a handful of responses received after this time, but they were not included in this analysis.The breakdown of response sources was as follows:

Level

Responses

Response rate (%)

Perlbuzz

57

37.3

Mailinglist

77

50.3

Perl mongers

8

 

Missing

11

 

Total

153

 

The questions in the questionnaire were as follows:

  • What country are you in?
  • How many people are on your team?
  • How many of those people are writing code with Catalyst? If there are non Catalyst coders on your team, how many of the whole team would you like to be writing Catalyst code?
  • How many people using Catalyst on your team are subscribers to the Catalyst mailing list?
  • How many people writing Catalyst code on your team use the #catalyst IRC channel on irc.perl.org?
  • What do you see as potential for growth of Catalyst in your organisation? How many people do you think will be using Catalyst in your organisation in 12 months? In 2 years?

Breakdown by country

Country

Count

Percent

US/span

48

31.4%

U/span

26

17.0%

Austria

9

5.9%

Australia

9

5.9%

Canada

8

5.2%

Germany

7

4.6%

Japan

4

2.6%

Norway

4

2.6%

Spain

4

2.6%

France

3

2.0%

Italy

3

2.0%

Portugal

3

2.0%

Russia

3

2.0%

Belgium

2

1.3%

Brazil

2

1.3%

Romania

2

1.3%

Agrgentina

1

0.7%

China

1

0.7%

Guatemala

1

0.7%

Ireland

1

0.7%

Malaysia

1

0.7%

Netherlands

1

0.7%

Slovenia

1

0.7%

South Africa

1

0.7%

Sweden

1

0.7%

Switzerland

1

0.7%

Taiwan

1

0.7%

Thailand

1

0.7%

Ukraine

1

0.7%

Missing

3

2.0%

Total

153

 

Distribution of total team size

Note that this question was answered as the total number of programmers on a team.

The average team size for Catalyst coders was slightly higher for people responding from Perlbuzz than from the Catalyst mailing list, indicating a lower level of community involvement for individuals from larger organisations. Correspondingly these are also people more likely to buy books.

Number of team members

Count

Percent

1

31

20.3%

2

17

11.1%

3

22

14.4%

4

16

10.5%

5

17

11.1%

6

7

4.6%

7

10

6.5%

8

8

5.2%

9

2

1.3%

10

8

5.2%

> 10

15

9.8%

Total

153

 

The median number of members of the team was four members with 50% of responses between 2 and 7 members.

Percentage of catalyst coders

The average proportion of catalyst coders on the team was 65% (standard deviation 33.9). Excluding teams of 1 (sole traders), the average proportion of Catalyst coders was 58% (standard deviation 32). This indicates quite a high usage of Catalyst across all teams, and its viability as a platform for small to medium sized teams.

Desire for more catalyst usage.

Three respondents indicated that they would like fewer people using Catalyst in their team. This may well represent misunderstanding the question. Of the remaining 92 respondents that had usable data for this question, 25% of respondents would like their team of Catalyst programmers to remain the same size. One quarter of respondents would like to see their team grow by up to 50%. A further quarter indicated that they would like to see growth of Catalyst coders increased usage of Catalyst in their team by between 50 and 125%, and the remaining group would like to see very high increased usage of Catalyst—between 125 and 900%.

This indicates that Catalyst is popular among its users, and that there is generally a desire to see quite high growth of catalyst usage among existing programmers in their team.

Mailing list subscribers.

Number of subscribers in team

Count

Percent

Cumulative Percent

0

24

15.8%

15.8%

1

87

57.2%

73.0%

2

24

15.8%

88.8%

3

9

5.9%

94.7%

4

5

3.3%

98.0%

5

1

0.7%

98.7%

7

2

1.3%

100.0%

Total

152

 

 

So 88% of responses indicated that two or fewer people in the team were subscribers to the mailing list. There is little difference in these figures if one-person teams are excluded.

This data with the team size allows us to estimate the number of Catalyst users per subscriber to the mailing list. First we exclude all team sizes above 15 as outliers. This leaves the mean team size as 4.4 members (95% confidence interval of 4.86 to 4.92). For this group we know the mean number of mailing list subscribers is 1.3, so 4.4/1.3 which indicates on average 3.39 Catalyst programmers per mailing list subscriber, or 3390 users of Catalyst across the mailing list.

For the 24 non-subscribers, the mean team size was 5.95. So as a minimum estimate, this represents another 150 users. Looking at it another way, the number of users can be inflated by a further 50% at least. The figure of 50% is taken as a conservative correction for non-response bias. 20% of our sample were non-mailing list subscribers, but as these individuals by definition are harder to reach (and more likely to be members of larger teams concurrently), it seems reasonable to inflate this estimate substantially. This is a conservative estimate because our ability to reach non-subscribers was limited by survey distribution channels and the relatively short turnaround. Therefore we are confident that a 50% correction for non-response bias is a conservative estimate, and puts the number of Catalyst programmers at around the 5000 mark, although we need to stress that this is a conservative estimate.

IRC Channel statistics

We know that there are about 175 regular IRC channel members. The survey includes 130 of these subscribers (although this is an overestimate as some occasional users would also have indicated membership). This is more evidence that our non-response bias correction of 50% is a conservative figure, as IRC regulars are likely to be committed members of the community.

Desired growth of catalyst coding

The responses to this question are difficult to interpret. However after 12 months the mean desire for growth of catalyst programmes employed in each team is around 200%. 14 respondents did not answer the 12 month growth question, and only a further 4 indicated that they expected no growth. This indicates a strong potential for the growth of usage of Catalyst. The 2 year question was not used due to the long term projection, and the consistency with the 12 month growth projections.

Miscellaneous comments

There were a few comments that indicated that the shortage of good printed resources are a problem for Catalyst. In general comments were supportive for a new book, and there was a tendency for larger teams to indicate the desire for several copies of a good quality book.