Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

export spss syntax => file not found

More
10 years 7 months ago - 10 years 7 months ago #116453 by Chuanse
Hello,

It seems that exporting survey results to spss is an issue on my server.
Exporting the .dat file works fine, but when we try to export the syntax file Firefox says: File not found
Chrome says: RR_INVALID_RESPONSE

I see no problems in my apache2 log...

The most weird thing is that it seems to be a specific survey that gives problem. Others are working fine. The survey is rather large. Could that be an issue?

Could someone help me out here? Google is not telling me a lot about this type of problem :(

Thank you

Version 2.05+ Build 141229 on Debian Linux wheezy with MySQL
Last edit: 10 years 7 months ago by Chuanse.
The topic has been locked.
More
10 years 3 months ago #121037 by yinsh
Hi, I have the same issue. I'd like to raise the same quesion if you don't mind. B)

I tried to export the spss syntax file, but Firefox says File NotFound, Firefox couldn't find the file on https://lime-server/index.php?r=admin/export/sa/exportspss/sid/xxxxxx. And Chrome says ERR_INVALID_RESPONSE, the page I require is temporarily not found or removed permanently. It happens when I try to export spss data file as well.

Since you mentioned your survey was quite large, I enlarged the server's memory limitation and max_post_size. It didn't work out. AND, my survey is quite small with 26 quesions.

The spss exportation will generate a file to be downloaded. So I guess maybe there's something wrong with the tmp directory which the tmp file should be in. Of course there is possibility that the system couldn't generate a spss syntax or data file at all. BTW, my server can export a .lss file all right.

I've checked the SPSSGetQuery and SPSSFieldMap functions, but I don't think they are the cause.

The survey that couldn't be exported was imported into limesurvey using a .lss file from a totally diferent limesurvey system in a different server. The one that couldn't export the file is 2.05+ Build 141210 and the original one that can export is 2.05+ Build 140520, where the .lss file is from.
The servers use Unbuntu 12.04. Apache/2.4.4 (Unix) PHP/5.4.14.Mysql 5.5.31.

I'll be really grateful if someone can help me with this!
The topic has been locked.
More
10 years 3 months ago #121065 by holch
I would guess that the export function runs in some kind of limit (script time or script memory size). You should have a look at that on your server. What are the values for this in your PHP installation?

Help us to help you!
  • Provide your LS version and where it is installed (own server, uni/employer, SaaS hosting, etc.).
  • Always provide a LSS file (not LSQ or LSG).
Note: I answer at this forum in my spare time, I'm not a LimeSurvey GmbH employee.
The topic has been locked.
More
10 years 3 months ago - 10 years 3 months ago #121106 by yinsh
Thank you for replying!
I have listed some of the settings in php.ini I think related.

File Attachment:

File Name: php.rar
File Size:19.87 KB


max_execution_time = 300
max_input_time = 600
memory_limit = 256M
register_argc_argv = Off
output_buffering = 4096
post_max_size = 100M
default_socket_timeout = 60
mysql.cache_size = 2000
mysql.max_persistent = -1
mysql.max_links = -1
mysql.connect_timeout = 60

The survey that I can't export has 26 questions, while there's another survey that has nearly 3000 quesions. I've changed the max_input_var to 2500 to make csv exportation running properly, but this didn't help with the spss.
Thank you! :cheer:
Last edit: 10 years 3 months ago by yinsh.
The topic has been locked.
More
10 years 3 months ago #121107 by Mazi
At "Plugins" check if the SPSS export plugin is enabled.

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
More
10 years 3 months ago #121122 by yinsh
Thank you for replying!

Since I didn't find SPSS plugin, I've enabled R and STATA. Like this:



I restarted apache and this didn't help. Firefox still says file not found.

BTW, I installed another limesurvey in anoter server, which has plugins like this:


And this system works all right.

I upload the export.php file if needed. Actually we are doing some development with limesurvey to make it work offline and upload the surveys back to the server. It's the server side that has the issue. We changed the import functions a lot for importing offline surveys, but didn't change the export function. I'm not sure if there's a connection. I upload the export.php as well.

File Attachment:

File Name: export.rar
File Size:10.48 KB
More
10 years 3 months ago #121127 by Mazi
I am pretty sure that this is related to the changes you made to the code. I have never seen such an error and if it works fine on other systems, this indicates that it is a very special error which probably is not related to server config but to code changes.

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
More
10 years 3 months ago #121129 by yinsh
We changed the vvimport function and add our own functions related to ftp upload in the dataentry.php. We ketp the original functions unchanged.
I didn't expect that will influence the functions in the export.php file because we didn't change it at all. But you do remind me of something about the database modification. We changed the lime_survey_(sid)' table structure by adding a few columns for our own needs. I'll see what I can do. Thanks a lot for reminding me!
I've tried my best to keep limesurvey the way it is, but I'm not the boss. :( But I think it's an amazing thing to make limesurvey more than on-line though it's against limesurvey's goal, right? :laugh:
The topic has been locked.
More
10 years 3 months ago #121136 by Mazi
Can you tell us a little more about your goals and developed features? Maybe you can open a new thread and link it here?
I remember lots of other users who may be interested in such an offline feature.

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
More
10 years 3 months ago #121296 by yinsh
Of course, I'd like to!
I'm not sure if this is what you expect, so let me describe our system here first, please. I will open a new thread if nessasery.

Accurately, we are working on a set of system consisted of an on-line server end for the supervisors to monitor the interviewing progress in a way and to collect data day by day, many off-line ends for the interviewers to do interviews separately and to upload their surveys when they can get on line, and an ftp server for file storing and transferring between the on-line server end and the off-line ends. The server end and the off-line ends are both based on Limesurvey. We add some file transferring functions between the two ends and are heading to develop some management functions on the server end. Now it's a very immature version. We try to use Limesurvey's own functions as much as possible to limit the interference to the whole.
The topic has been locked.
More
10 years 3 months ago #121298 by Mazi
Thanks for providing these details. Are there any plans to set the additional script up as Limesurvey plugins? That way you are future save when updating later.

Can you describe how the data collected offline gets transferred to the online server?
Is this process automated based on detecting an available Internet connection?
Do users need to select the to be transferred data sets manually and press a button?
How do you make sure no duplicates are transferred?

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
More
10 years 3 months ago #121303 by yinsh
Thanks about your advise on the plugins. For now we haven't planned to do so. But as we understand Limesurvey better, I think it would be a better way to set up a plugin.

We DO transfer duplicates! :lol:
The supervisors need all the surveys, completed or not completed, to monitor the interviewers' performance and give them the proper instructions. So the surveys will keep transferring back to the server end until they are completed. If I want to know if one survey is in progress or how many days the interviewer spend on one interview, the comparable duplicates will help. Since our survey is quite long, it's difficult to finish in one day. For quality control.

Of course we don't want too many duplicates. Our interviewers transfer data once a day. So this makes it impossible to transfer data automaticly. We give them a warning if they try to transfer data when there's no Internet connection.

The interviewers never see the administrator interface. When they want to transfer data, there's button to press. As simple as that. No selection of data. We add a function based on vvexport to automaticly upload the data to the ftp server for data storing. On the server end, all the data uploaded to the ftp will be inserted to the database using an added function based on vvimport.

Besides surveys, the off-line end can update samples as well. So all the interviewers will do is to press the update button to update samples, do the interview, and press the upload button to upload data. He/She may also check which samples are completed or not. But they don't have access to the administrator interface.

On the server end, we identify the very survey using the off-line ends and keep the other surveys using limesurvey as usual. So the server end is for both off-line surveys and on-line surveys.

There're some areas with bad internet connection, so maybe it's one of the suitable ways for our situation.
The topic has been locked.
More
10 years 3 months ago #121337 by Mazi
Thanks for this additional information. That sounds really interesting.

A few follow up questions if you don't mind:

1. How (technically) do you detect if an Internet connection is available?

2. Do you use a daily cronjob to import the uploaded VV files?

3. I still don't really understadn how you make sure that e.g. data set "A" (completed yesterday) doesn't get uploaded again at todays upload.
Do you only upload those data sets being edited the current day?

4. Does updating samples mean that they load a list of new participants from the server, like a CSV file of new token data?

Thanks for sharing your approach.

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
More
10 years 3 months ago #121370 by yinsh
I'd love to share information and am open to all the opinion and advices.
1.Try to open a web page file, such as www.google.com , using fopen( ) function. If we can open the file, then the internet is connected, otherwise, no connection.

2.I really appreciate you mentioned the cron job! For now, we've only completed the functions to meet the basic needs. The supervisors will have to import the vv files manually. Of course we rewrite the dataentry.php file so that they can import many vv files just like importing one. We have already planned to automaticly import vv files using shell in the next version.

3.We can't just upload the current day's surveys because we can't be sure that the interviewers can upload their surveys everyday, although the off-line end will pop up an alert when the interviewers haven't done the uploading operation in the last 24 hours.
So here's our solution: we modify the lime_survey_(sid) table by adding some useful columns, one of which is to record whether the certain row needs to be uploaded in the next uploading operation. Let's name the column A. A is either 0 or 1. 0 is the default value, meaning this row of data needs to be uploaded. 1 means that the row has already been uploaded in completion and doesn't need to be uploaded any more. Now you can tell, the newly inserted surveys, the uncompleted surveys and the not-yet-uploaded completed surveys are all set to 0. Only the ones that are completed and just uploaded will be set to 1, so in the next uploading the row wouldn't be uploaded again because we only upload those with A being 0.

4. We update the token table in the off-line end. Our survey is not anonymous and has a token table. The supervisors manage the token table on the server end. They generate a .sql file from the token table and upload the file to the ftp server when there are samples to be updated on the off-line ends. The interviewers will just click the update button and the .sql file will be downloaded to their systems to update their token tables backstage.

We are doing this for our own needs. I'd really like to know what kind of off-line limesurvey do you expect if you don't mind.
The topic has been locked.
More
10 years 3 months ago #121376 by Mazi
Again, thanks for your feedback.

@1: That's the approach I had in mind as well.

@2: For projects like this I use cronjobs a lot. It really helps to automatize stuff like such daily jobs.

@3: That additional column for noting the upload status makes sense. Did you think about using an additional table instead? Adding columns to the main "lime_survey_12345" table may break other features.
All you need when using a new, separate table is SID + Data set ID + status column.

@4. Do you create a single file for each interviewer?

@"I'd really like to know what kind of off-line limesurvey do you expect if you don't mind.": Sure: Our company deals with ~100 survey projects for dozens of customers each year. Some of them have similar requirements:
- doing surveys offline
- uploading data if Internet connection is available
- keeping data in sync

Most of them decided to not spend any additional money on coding such features you mentioned. Instead, they do all that uploading and data export/import stuff manually.
As an IT guy, I really hate seeing this because I know that when spending just a little time (and money), such a process can be automatized nicely, as you have proven.

If you are interested in more details, you can drop me a note at marcel.minke(at)limesurvey.org

Best regards/Beste Grüße,
Dr. Marcel Minke
survey-consulting.com
offlinesurveys.com
Feel free to contact me by email for professional LimeSurvey support!
The topic has been locked.
Moderators: holchtpartner

Lime-years ahead

Online-surveys for every purse and purpose