Seafarers show measurable progress using video based online learning program

We have seen that 50 to over 95% of the participants were below the level required by the involved Dutch and German shipmanagement companies. The gaps were on average 12 - 26 points (on the CEFR scale of 1 - 100), two of the three groups showed improvements of 10 or more point going through the regular online video program. In this article we share these and other outcomes of our research. - First published June, 2017 -

Helping international teams and crews learn and change faster

Our aim as a company  is to help organisations with international teams and crews change and learn faster. As an online learning company we monitor progress and results of our students continuously to find out if we actually succeed in helping our clients and their workers in learning new things (gain knowledge, improve skills). Speed in acquiring new knowledge and skills is important because our clients operate in complex, dynamic environments. So we do not only measure if people learn new things, we also try to design effective, often pretty short programs and try to find out which of our learning interventions work best. We also personalise our programs continuously, using the data we collect. In that way we can help participants find the shortest road to reach their learning destination.

Effectiveness of online learning in improving English language skills of seafarers (and other maritime professionals)

In 2017 we have done our most extensive research effort so far into the effect of our English language programs on the English language proficiency of maritime workers. So we collected data of almost 650 maritime professionals (most of them seafarers) that have been learning with us in the last two years. We have had more participants than this of course, but we tried gathered the data of students that went through our regular programs in the most ‘normal’ way, so we we would be able to compare things and draw sensible conclusions. In this article we share some of the conclusions of our research.

Digging into the data was essentially a humbling experience. Even though we collect a huge amount of data continuously, it’s certainly not easy to draw simple conclusions about our methods and actions, that have been developed not too long ago and have undergone a lot of changes, as we are a startup and innovator pur sang.

Diving into our data taught us that there is a lot more that we want to learn, that we must work even harder on collecting (enough of the) data in consistent ways and in accessible formats and that we want to get even better at asking the right questions and designing strategies to come up with the answers after some time.

What kind of maritime professionals were in the group we researched?

The data we looked into were the results of almost 650 maritime professionals that were enrolled in our maritime English program. In this research we looked into the data of Russian and Filipino seafarers. We also had some groups of office workers, mostly Dutch and German, and there was a small group of seafarers that were from other - mostly Eastern European - countries. We have also collected a nice amount of data about Chinese seafarers that have studied with us but we have done a separate study into those data (you can read this article about a project to improve English proficiency of a group of Chinese seafarers). The numbers of participants from China are smaller than the those from Russia and the Philippines and there are some other differences that make it more difficult to compare the results of the Chinese seafarers with the results of the students included in this group of almost 650 mostly Russian and Filipino crew members.

Gaps between required level and measured entry level

All participants are tested first when they enroll in the program. Our client let’s us know what the required level of each participant is.

We call the difference between the level required from a participant and his actual level at entry of the program his ‘gap’.

The Russian seafarers were in most cases below the level required (of one group 96% was below required level, of another smaller group 90% was below the required level). Of the two Filipino groups respectively 50% and 58% of the participants tested at entry were below the level required by the principal.

The average gaps of the tested groups were (gap = required level - tested entry level in points on CEFR scale 1 - 100):

• Russian group 1: 26 points
• Filipino group 1: 16 points
• Filipino group 2: 12 points

This research focusses on participants that are confronted with pretty tough required levels by the principal. Officers are required to have a CEFR level of C1, just like office workers. CEFR, short for  Common European Framework of Reference for Languages: Learning, Teaching, Assessment, is a European Standard, more information can be found here..The lowest level required from the participants included in this research is A2 (required for ratings, support), for most other seafarers either B2 or in some cases B1 is required.

In the Russian group there was a relatively high percentage of officers, compared to the groups from the Philippines.

Positive, measurable results in learning English online

The data show that participants show measurable progress when we test them after finishing the first part of our online language program.

When seafarers participating in the program have test scores below the level required they are assigned courses that match their level and can help them improve as quickly as possible. These courses are general English courses, aimed at helping them improve their general English proficiency.

Participants are monitored closely. We collect data and in that way measure progress and identify ways to help them accelerate wherever possible. In our method (and that of our partner EnglishCentral) there are several ways to make clear where specific learning interventions can help participants improve specific skills. We will speak  a bit more about this later in this article.

When a participant has finished the assigned courses and has worked on extra learning activities that are suggested during the program he will again be tested to measure progress. The test is done by Skype. Like the entry level test it is done during a live conversation with an English teacher. The progress by participants of the three groups we looked into were

• Russian group 1: 11 points
• Filipino group 1: 10 points
• Filipino group 2: 6 points

The seafarers that went through a first set of courses that were assigned to them after the entry level test have in many case closed a significant part of their personal gap.

Regular exercise for an extended period makes a (big) difference

The courses are carefully chosen to match the entry level and help improve substantially in two months time. That requires regular exercise. We ask participants to exercise at least three times a week for at least three quarters of an hour to an hour. Regular exercise is necessary. The data show that it is best to take at least 7 or 8 weeks and exercise several times a week.

Sometimes participants finish their courses quickly by exercising several hours at a time for several days in a row. Those students often show less progress when tested. This was especially the case in Filipino group 2.

Students that exercise regularly during an extended period show better results at the second test (what we call the ‘level up test’).

More about the entry level the participating seafarers (and other maritime professionals)

During the entry level test (taken by skype) and the level-up test the teacher will rate the overall English language proficiency but will also specifically evaluate his:

• Comprehension (listening)
• Grammar and vocabulary
• Communication (speaking)
• Pronunciation
• Fluency

In this research we’ve seen that the level of comprehension (listening) is in general higher than the average (overall) score. Pronunciation and fluency are often the lowest subscores of participants. This is something we see in the maritime world most of the time. Grammar and vocabulary vary a lot, we suspect that this subscore is most related to the educational background of the participant. Other skills, especially comprehension (listening) skills are more easily further developed in practice.

Our video-based methods and the speech technology and personalised feedback systems behind EnglishCentral are designed to help improve pronunciation and fluency. The test results show that this leads to measurable results.

About English vocabulary

Our most demanding clients require a C1 level for officers. People with that level have an English vocabulary of between 5,000 and 8,000 words. For A2 (the level required for ratings by most of our clients) the average vocabulary will be between 1,400 and 2,200 words.

To put this in perspective:

• A well-educated native English speaker has a vocabulary of 20,000 - 30,000 words
• Non-native speakers have much lower vocabularies, even if they are considered to be very good at English. Only people that stay in English speaking countries for some time have vocabularies above 10,000 words
• A native speaker has a vocabulary of around 5,000 words at the age of 5
• Non-native speakers with (the right) vocabulary of 3,000+ words can be called fluent. With a kernel lexicon of 2,400 words someone can already understand 75% of what he reads. But of course that is pretty theoretical, as most communication in practice is not in written form
• A non-native speaker that has finished secondary school and got good grades should have a vocabulary of 4,500+ words

In Spot On Learning’s maritime English program we start by working on the General English vocabulary. More about the methodology behind building the vocabulary (and the personalised study list of words) can be read in this article on this website: Online methods to improve English communication skills of international teams and crews.

The next step is to build a specific maritime vocabulary. We start that part of the program when the participant has reached the required level of general English.

Our research into the 650 seafarers that have gone through our program that their vocabulary grows measurably during the program. The personalised study list feature with quiz functionality which is part of the EnglishCentral platform helps our participants building their vocabulary, both a more general vocabulary and a specific maritime vocabulary with words taken from IMO’s SMCP list.

Lessons learned: the importance of motivation to learn and speak English

Our experience and research into this group of around 650 maritime professionals gives us a lot of information about motivation of the participants. What we have learned is that it is important that the seafarers are informed about the ‘why’ of the English language program. While going we have experimented a lot with content and tone of emails and other communication. During the onboarding process we ask questions like

International laws, regulations and most shipping companies require that all seafarers speak English at a certain level. Did you know this?

• Did you know that [name of principal/employer] requires that all seafarers and office staff members speak English at a certain level?
• How do you feel about improving the English communication on board?
• Do you think it is possible to improve your English online?

We also have questions about how ‘digital and online’ they are outside their professional life. By asking questions and interacting in other ways we try to make our participants think about their own motivation, their own interests to invest in the program by their own time and efforts.

Sometimes we see that participants have problems getting started and building a habit of exercising regularly. The research shows that if participants are not informed well about the why and how of the program they often drag their feet.

In general we learned to maximise our efforts at the beginning of the program. If people start shortly after being invited, find their way quickly (with or without our support), use personalised features early they go through the program smoothly in most cases. Participants that procrastinate or get into trouble early often have more problems throughout the problem, need more support, progress slowly and show less favourable results.

Lesson learned: it’s the network, stupid!

We also found out that we - as a provider of the English language (and other) programs for maritime professionals - can not do all the motivating. We have to build a network of stakeholders and colleagues because they are influencers of our students. The principal can clearly communicate what they require from their crews and other workers. The crewing agent certainly has an interest (seafarers) and can influence the student by communication about the why. We see that both the ‘carrot’ and the ‘stick’ can be effective. Also we have learned that there are many cultural aspects of motivation that can make a big difference.

We share information about progress and results of our participants with the network of stakeholders. By doing this we build a supporting network around the participants.

Lesson learned: personalisation works

One of the things that seems to influence motivation greatly is personalisation. The level test and level-up test are both conversations with a real teacher. The participants like this approach in most cases, though they can also be somewhat nervous about the test. After the test the participants are invited for courses that match their level. When they have spoken enough videos two personal features of EnglishCentral start working. Their personal pronunciation dashboard is generated and they can start using their personal study list to expand their vocabulary. We tell the participants how to use these features and we see that this motivates them to be more active on the system. We also see that feedback we give helps motivate them.

Next steps

In the past two years we have learned a lot about how we choose interventions that help, how we can explain important workings of the system and personalised functionality. We have learned what motivates and stimulates most seafarers to work on the program and yes, we also have learned what NOT to do.

Continuous improvement is important on board ships but it is certainly also very important for us to help international crews and teams change faster. We collect enormous amounts  of data and monitor as much as we can. In our team we try to take the time as often as possible to draw conclusions and improve things. This research has helped us greatly, hopefully it helps you as a reader that we shared some of it. Please contact us if you have questions about our research and our work.