Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

Identify Geo-Location Based on IP address

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #226777 by marcgold
Please help us help you and fill where relevant:
Your LimeSurvey version: Version 3.28.3+220315
Own server or LimeSurvey hosting: Own Server
Survey theme/template: Fruity
==================
Hi

Wondering how easy this is to achieve although I know its not 100% accurate.   We are running a lot of market research public surveys per month and on reviewing the IP Addresses recently it has become obvious that we are receiving a larger number than usual of fraudulent applications from out of country.  

We record the IP addresses when the surveys are taken.  What I would like to do is use this IP address to run a GEO locate via an API and deposit the responses into a hidden question in the results so the team reviewing the answers can see straight away the town: country of the connection.   We know this is not 100% as some use VPN's but it should go someway to help the situation.

Does Google have an API for this?   I have come across IPStack.com but at $600 a year this seems expensive (500k requests)  - we are doing at least 100k applications a month.

Does anyone have any scripts, plugins ideas on how to solve this.

Thanks in advance

Marc Gold 

Please Log in to join the conversation.

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 1 month ago #226781 by holch
Replied by holch on topic Identify Geo-Location Based on IP address
There are so many of those services out there. A search for IP to Geolocation gives me a lot of results, e.g. ipgeolocation.io/ , www.geoapify.com/pricing , etc. I have no idea how good they are.

But $ 600 for 500k sounds pretty OK for me. this is about US$ 1,20 per 1000 consultations. I mean, if you run 100k online interviews per month, your company probably has probably quite a high volume of sales as well. I think 1,2 dollars per 1000 interviews should be easily covered.

What I did for some projects was using a hidden Google Maps question in Limesurvey that would write the GPS coordenats into a hidden text question.

Worked pretty well on a smartphone, on the computer the margin of error is bigger (uses different ways to determine location, not GPS). But I think for your case it won't really work, because the "respondent" needs to allow the geolocation.

For us this was OK, because the questionnaire was applied by our interviewers and the location was more for our documentation and a way to show the end client that the interviews where done at the right location.

And with the amount of consultations, you probably would need the Google Maps API anyway.

Help us to help you!
  • Provide your LS version and where it is installed (own server, uni/employer, SaaS hosting, etc.).
  • Always provide a LSS file (not LSQ or LSG).
Note: I answer at this forum in my spare time, I'm not a LimeSurvey GmbH employee.

Please Log in to join the conversation.

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #226791 by marcgold
Replied by marcgold on topic Identify Geo-Location Based on IP address
Thanks Holch
So been doing some checking, and used the  ipgeolocation.io  service for testing.    
So what I have done is created a Text question to capture the output,  initial testing is sort of working but with some issues when you hide the question as the JS doesn't fire.   I have tried hiding the question using JS, but the submit button doesn't hide so the user has to click it to continue.   I have tried an autosubmit,  but this is not working either.   

Here is the code i'm using  - can you suggest how to do this so it works and the question never shows (or the submit button).
Code:
<script src="https://cdn.jsdelivr.net/npm/ip-geolocation-api-jquery-sdk@1.1.0/ipgeolocation.min.js"></script>
 
<script>
//Hide the page with JS
$('#question{QID}').hide(); 
 
    // On call to IPGeolocation API on each page during a user's visit, API response will be served from sessionStorage after the first page.
    _ipgeolocation.enableSessionStorage(true);
 
    var ip = sessionStorage.getItem("ip");
    var country_name = sessionStorage.getItem("country_name");
    var country_code2 = sessionStorage.getItem("country_code2");
    var city = sessionStorage.getItem("city");
            
    if (!ip || !country_name || !country_code2 || !city) {
        _ipgeolocation.makeAsyncCallsToAPI(false);
        _ipgeolocation.setFields("city,country_name,country_code2");
        _ipgeolocation.getGeolocation(handleResponse, "************************************");
    }
 
    function handleResponse(json) {
        ip = json.ip;
        country_name = json.country_name;
        country_code2 = json.country_code2;
        city = json.city;
    }
 
$(document).ready(function(){
        var whatiwant = "this is what I want";
        $("#answer{SGQ}").val(city+" ,"+country_name+", "+country_code2).trigger('keyup');
    });               
// Try and auto submit the page 
$(document).getElementById('#limesurvey').submit();
  </script>

Thanks
Marc

Please Log in to join the conversation.

  • tpartner
  • tpartner's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 1 month ago #226792 by tpartner
Replied by tpartner on topic Identify Geo-Location Based on IP address
Always run surveys in group-by-group mode so you can hide or combine questions.

Hide the question with a CSS class "hidden" and place it in a group with another question.

Cheers,
Tony Partner

Solutions, code and workarounds presented in these forums are given without any warranty, implied or otherwise.
The following user(s) said Thank You: holch

Please Log in to join the conversation.

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #226822 by marcgold
Replied by marcgold on topic Identify Geo-Location Based on IP address
Hi Tpartner

Thank you, that makes perfect sense on how it should be used in this environment.

However, based on some usability testing we decided some time ago to move away from By Group to one question at a time. The background is the surveys are used as registration process within a market research environment to identify prospective participants. We have identified the following:
  • Over 80% of registrations come from a mobile device so small screen  (people don't like scrolling - I know go figure )
  • Data accuracy is improved if a user is only shown one question
  • Less chance for people to change answers to fit the scenario
  • Partial data collection is improved
  • Lower number of fake / fraudulent applications
So I would like to look at working up a solution that would work well for every use case  regardless of All in one, Group or One by One.    Im thinking that JS is probably not the best way to go as this is too manual.    Im thinking either to create a plugin that can work or directly coding this into the template via PHP.  

What do you think?

Marc

Please Log in to join the conversation.

  • Joffm
  • Joffm's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 1 month ago #226824 by Joffm
Replied by Joffm on topic Identify Geo-Location Based on IP address

Always run surveys in group-by-group mode so you can hide or combine questions.

and

to move away from By Group to one question at a time

This is not a contradiction.

We often recommend to display "group by group" with just one question per group.
Here you have the advantage to "combine questions" by javascript.
And the look&feel is like "question by question".

Joffm
 

Volunteers are not paid.
Not because they are worthless, but because they are priceless

Please Log in to join the conversation.

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #226827 by marcgold
Replied by marcgold on topic Identify Geo-Location Based on IP address
Thanks Joffm,

I agree that this is not a contradiction and of course having questions Group by Group provides some advantages. But one needs to also consider the usability and admin overhead in creating every question in its own group.

I can see the advantages of doing it this way, however, in our use case (which may be unique - although I'm sure its not) this method adds a technical overhead that is unwelcome.

To explain, as mentioned our use of LS is as a registration system for Market Research purposes. As a result we can create 3 - 5 surveys a day of varying length. Some short 10 Q's - sometimes 30 Q's. Complete with routing, relevance and conditions. Each survey on average takes 60 - 120 mins to build depending on complexity, and adding a new group for every question adds 30 - 60 min build time to each survey. On a bad day (5 x complex) this would add an extra 4 - 5 hours work. Now multiply that out over a week, month etc...

This is why I need to find a better solution.

So what I need is for the survey to run Q by Q (This works for us). The ability to add new Q's quickly, with the built in "Add New Q" (in the same group).
But not to loose focus on the fact we need to GEO Locate based on IP Address and insert it into the data set somewhere.

Thinking logically (I hope). I feel the best solution would be to run the code in PHP (maybe as a plugin), and to create 1 or more system variables that can be inserted into the data set (similar to IPADDR:). This way it is usable by anyone in any scenario that need this or similar functionality.

Don't get me wrong, I love LS and always have done. I have been using it in for a very long time, but 15 years ago there was nothing out there that came close. These days, this is not such the case and there are many solutions out there with similar functionality that is much easier to use and build from a none coders perspective.

But I suppose that why we rely on very clever people like you :)

So I have a pretty clear idea now of what I want from a plugin that would benefit others but no idea how to even start to build it.

Would anyone like to help??

Regards
Marc

Please Log in to join the conversation.

  • Joffm
  • Joffm's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 1 month ago #226831 by Joffm
Replied by Joffm on topic Identify Geo-Location Based on IP address

and adding a new group for every question adds 30 - 60 min build time to each survey.

Sorry, then you do something wrong.
To create a group takes about 5 seconds.
And if you create all questions in one group or each in a different will not be such a difference in time.

The ability to add new Q's quickly, with the built in "Add New Q" (in the same group).

Right, but the selection of the group in the drop-down after having clicked "Add new question" again is not really time consuming.

And even if there is an existing survey.
Create all groups, one after the other.
Then replace the questions with the organizer. Again only a few minutes.

But - of course - it is your decision to do it question by question.

Joffm

By the way:
To answer your question about the submit button.
$('#ls-button-submit').trigger('click');

And now let's continue withe geo-location

Volunteers are not paid.
Not because they are worthless, but because they are priceless

Please Log in to join the conversation.

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #226838 by marcgold
Replied by marcgold on topic Identify Geo-Location Based on IP address
Joffm

Again thank you, i really appreciate your insight and the help with the Click Submit fix...   In fact your help has enabled me to put the final pieces into the puzzle and I know have a perfectly working solution from all perspectives - user and admin.
I still think having this as a plugin would better server the community so will publish a design brief and see if anyone wants to help.   

But for now the solution:
  • Works in Q by Q mode   (if anyone is interested REMOVE the Auto submit code at the end if not in Q by Q mode as it will break it)
  • Does not display Q or button
  • Moves straight on to the next question
One of the issues was the speed of the JS firing - it took a few seconds where it rendered the button and then removed the button from view.  Removing the CSS manipulation from JS and putting it into a simple <Style></Style>   meant it wasn't rendered to the screen in the first place.

Here is the final code if anyone needs this.   Works in LS 3.28
Hope this helps someone.
Code:
<style>
#ls-button-submit {
       display:none;
}
#question{QID} {
       display:none;
}
</style>
 
<script src="https://cdn.jsdelivr.net/npm/ip-geolocation-api-jquery-sdk@1.1.0/ipgeolocation.min.js"></script>
 
<script>
 
    // On call to IPGeolocation API on each page during a user's visit, API response will be served from sessionStorage after the first page.
    _ipgeolocation.enableSessionStorage(true);
 
    var ip = sessionStorage.getItem("ip");
    var country_name = sessionStorage.getItem("country_name");
    var country_code2 = sessionStorage.getItem("country_code2");
    var city = sessionStorage.getItem("city");
            
    if (!ip || !country_name || !country_code2 || !city) {
        _ipgeolocation.makeAsyncCallsToAPI(false);
        _ipgeolocation.setFields("city,country_name,country_code2");
        _ipgeolocation.getGeolocation(handleResponse, "ENTER_YOUR_OWN_API_KEY_FROM_IPGeolocation.io");
    }
 
    function handleResponse(json) {
        ip = json.ip;
        country_name = json.country_name;
        country_code2 = json.country_code2;
        city = json.city;
    }
 
$(document).ready(function(){
        var whatiwant = "this is what I want";
        $("#answer{SGQ}").val(city+" ,"+country_name+", "+country_code2).trigger('keyup');
 
// Remove the next 1 line if you are not using Question by Question Mode as it auto moves to the next question.      
                $('#ls-button-submit').trigger('click');
    });               
 
 
  </script>

Massive thanks to everyone that got us to this point 
Marc

Please Log in to join the conversation.

  • marcgold
  • marcgold's Avatar Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #226874 by marcgold
Replied by marcgold on topic Identify Geo-Location Based on IP address
So we have been working through an implementation for this and have extended the functionality a little.  Some of the data regards Proxy / VPN was missing from the original API.   Within 24 hours the provider IPGeolocation.io responded by releasing and pushing a new version of the script (V1.1.1) that included this all in an Array - Security.

There is some very useful additional information in here to help identify fraudulent applications like:
Threat Level:
Proxy: (True or False)
Proxy Type: (VPN / Socks etc)

Hope this is useful to others.  
We switched the Question type to Multiple Text - (only to make it more user readable in the end result for us)  But this will work on different question types but you will need to address the {SQA}'s slightly differently depending on which question types your using.

Working in LS 3.28

Set up your Question
  • Create Multiple Short Text Question (Ideally very first question but this is not so important)
  • Add the JQuery code below to the Question Text  (Remember to get your own API ID from ipgeolocation.io    up to 3000 calls per month is free. 
    (Note - if you wish to use the extended Security Array you must be on a paid for account which starts at $15 per month) 
  • Ensure Mandatory is false  (Any fields returned NULL will cause it to fail otherwise)
  • Ensure Always Hide Question: FALSE  (We will do this in CSS otherwise JS wont work as it needs to be rendered)
  • Save and Add Subquestions  For simplicity we used ID's of A1, A2, A3... (in our example these subquestions are City, Country, Proxy & Proxy Type)
Code:
<style>
/* Hides the Submit button from view*/
#ls-button-submit {
       display:none;
}
/* Hides the Question from view*/
#question{QID} {
       display:none;
}
</style>
 
<script src="https://cdn.jsdelivr.net/npm/ip-geolocation-api-jquery-sdk@1.1.1/ipgeolocation.min.js"></script>
 
<script>
 
    // On call to IPGeolocation API on each page during a user's visit, API response will be served from sessionStorage after the first page.
    _ipgeolocation.enableSessionStorage(true);
 
    var ip = sessionStorage.getItem("ip");
    var country_name = sessionStorage.getItem("country_name");
    var city = sessionStorage.getItem("city");
    var security = sessionStorage.getItem("security"); 
            
    if (!ip || !country_name || !city) {
        _ipgeolocation.makeAsyncCallsToAPI(false);
     // remove following line if not collecting additional security array available in paid for accounts only
        _ipgeolocation.includeSecurity(true);
        _ipgeolocation.setFields("city,country_name,security");
        _ipgeolocation.getGeolocation(handleResponse, "GET_YOUR_OWN_API_KEY_FROM_HTTPS://IPGEOLOCATION.IO");
    }
 
    function handleResponse(json) {
        ip = json.ip;
        country_name = json.country_name;
        city = json.city;
        security = json.security;
    }
// The following adds the data to LS Fields setup as multiple short texts 
$(document).ready(function(){
        $("#answer{SGQ}A1").val(city).trigger('keyup');
        $("#answer{SGQ}A2").val(country_name).trigger('keyup');
 
// remove next 2 lines if not using the additional security array data in paid for accounts.
        $("#answer{SGQ}A3").val(security.is_proxy).trigger('keyup');                
        $("#answer{SGQ}A4").val(security.proxy_type).trigger('keyup');                
 
// Auto submit the question once data is loaded
             $('#ls-button-submit').trigger('click');
    });               
 
 
  </script>


Enjoy 
Last edit: 3 years 1 month ago by marcgold. Reason: Adding LS Version
The following user(s) said Thank You: tpartner

Please Log in to join the conversation.

Moderators: tpartnerholch

Lime-years ahead

Online-surveys for every purse and purpose