How to download a file from the internet? [SOLVED]

Helper

On 06/10/2014 at 02:33, xxxxxxxx wrote:

I will try first sorting the #include's.
Why is it that it compiles and works just fine in Xcode but not in Visual Studio?

Helper

On 06/10/2014 at 03:18, xxxxxxxx wrote:

Ok, no amount of shuffling around makes the errors go away
I will try to make the curl stuff into a separate .cpp file now.

Helper

On 06/10/2014 at 05:33, xxxxxxxx wrote:

Good news and bad news.
The good news are that I no longer get the errors I was getting.
The bad news are that I now get new errors that I really don't understand:

1>Link:
1> Creating library C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\obj\PolyPaintTag\Win32_Release\PolyPaintTag.cdl.lib and object C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\obj\PolyPaintTag\Win32_Release\PolyPaintTag.cdl.exp
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_cleanup referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_perform referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_setopt referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_init referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\PolyPaintTag.cdl : fatal error LNK1120: 4 unresolved externals
1>
1>Build FAILED.
1>
1>Time Elapsed 00:00:05.36
========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========
1>Link:
1> Creating library C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\obj\PolyPaintTag\Win32_Release\PolyPaintTag.cdl.lib and object C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\obj\PolyPaintTag\Win32_Release\PolyPaintTag.cdl.exp
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_cleanup referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_perform referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_setopt referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>access_url.obj : error LNK2019: unresolved external symbol __imp__curl_easy_init referenced in function "bool __cdecl check_online(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >)" (?check_online@@YA_NV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@00@Z)
1>C:\Program Files\MAXON\CINEMA 4D R14\plugins\PolyPaintTag\PolyPaintTag.cdl : fatal error LNK1120: 4 unresolved externals
1>
1>Build FAILED.
1>
1>Time Elapsed 00:00:05.36
========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========

Like before, it all works fine in Xcode.

Helper

On 06/10/2014 at 07:56, xxxxxxxx wrote:

you should link to the curl library, in VS , project-> properties-> linker-> general -> Additional library directories , put /lib folder of the curl lib

and in the linker->input->Additional dependencies , put the curl library name, example:the_curl_library_name.lib

Helper

On 06/10/2014 at 09:15, xxxxxxxx wrote:

I found the source of the unresolved symbols problem I had with: **curl = curl_easy_init();

** I installed the 32 bit version of curl. And my C4D plugin was set to Debug | x64.
When I changed my C4D plugin to Debug | Win32. It works. And I can write curl code inside of my plugin's Execute() method now without any errors.
So I guess that's very important!!

I tested it using your small curl code Rui. And it seems to work fine.
I have not yet figured out how to convert the output of the code into GePrint compatible code yet:
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
But it does run. And it does grab a file from the internet without errors. When executed from my C4D plugin.

NOTE:
Since C4D plugins are basically .dll based programs.
You have to put the built libcurl.dll file in your main Maxon folder where the .exe files are.

-ScottA

Helper

On 06/10/2014 at 09:49, xxxxxxxx wrote:

But my plugins are for selling. Must I force people to place the libcurl.dll files next to the exe files?
Or is this just for compiling?
Also, to compile WIN32 and WIN64 versions must I install both versions of curl?!?

Helper

On 06/10/2014 at 10:17, xxxxxxxx wrote:

I'm pretty sure there's some plugin developers that sell plugins that require the users to add a .dll to the MAXON folder.
It's really not a big deal. Because the folder is easy to find.

Based on my results. I'm guessing that you'll have to install both 32 and 64 bit versions.
I personally wouldn't bother with 32 bit plugins anymore.

I still can't get C4D to return a proper result yet
This is the relevant code that's in my CommandData plugin:

#include <stdio.h>  
#include <curl.h>  
#include <iostream>  
#include <string>  
  
#include "c4d.h"  
#include "c4d_symbols.h"   
  
  
Bool SimplePlugin::Execute(BaseDocument *doc)  
{  
  CURL *curl;  
  CURLcode res;  
  char *usrpass="my_username:my_password";  
  char *buffer="1234567890";  
  
  curl = curl_easy_init();  
  
  if(curl)   
  {  
      curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com/robots.txt");  
      curl_easy_setopt(curl, CURLOPT_USERPWD , usrpass);  
      curl_easy_setopt(curl, CURLOPT_READDATA, buffer);   
      curl_easy_setopt(curl, CURLOPT_READFUNCTION, NULL);            
  
      //Perform the request  
      //res will get the return code  
      res = curl_easy_perform(curl);  
      String r = curl_easy_strerror(res);    
      GePrint(r);       //<--- Failed writing received data to disk/application   
  
      /* Check for errors */   
      if(res != CURLE_OK)  
      {  
          GePrint("curl has failed");   //<-- The console also prints this..curl has failed :-(  
  
          //Always cleanup   
          curl_easy_cleanup(curl);  
  
          return FALSE;  
      }  
  }  
  
  EventAdd();  
  return TRUE;  
}

Everything seems to be working. And I'm pretty sure it's reading the HTML file.
But I can't get C4D to use it yet.

-ScottA

Helper

On 06/10/2014 at 10:52, xxxxxxxx wrote:

After a quick spin around the internet.
I found a converter that will convert the data into strings.

This is the relevant code from my C4D CommandData plugin.
It grabs the text from a website using curl. And displays it to the C4D Console:

#include <stdio.h>  
#include <curl.h>  
#include <iostream>  
#include <string>  
  
#include "c4d.h"  
#include "c4d_symbols.h"   
  
  
  
static size_t WriteCallback(void *contents, size_t size, size_t nmemb, void *userp)  
{  
  ((std::string* )userp)->append((char* )contents, size * nmemb);  
  return size *nmemb;  
}  
  
Bool SimplePlugin::Execute(BaseDocument *doc)  
{  
  CURL *curl;  
  CURLcode res;  
  std::string readBuffer;  
  
  curl = curl_easy_init();  
  if(curl)   
  {  
      curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com/robots.txt");  
      curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);  
      curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);  
      res = curl_easy_perform(curl);  
      curl_easy_cleanup(curl);  
        
      //Convert the string to a String compatible with C4D  
      String convertedToC4D = readBuffer.c_str();  
      GePrint(convertedToC4D);  
  }  
  
  EventAdd();  
  return TRUE;  
}

phew!...this is too much like work.!Wink[URL-REMOVED]
Why do I feel like Dr. Frankenstein?

-ScottA

[URL-REMOVED] @maxon: This section contained a non-resolving link which has been removed.

Helper

On 06/10/2014 at 11:26, xxxxxxxx wrote:

So, I don't need to bother creating .cdl files anymore?
Why, oh why isn't developing and compiling stuff in VS as easy as with Xcode?!?!?

Helper

On 06/10/2014 at 12:33, xxxxxxxx wrote:

You still need the .cdl files and your usual plugin stuff.
The .cdl files are for running your plugin.
The libcurl.dll file is how C4D itself finds the curl libraries.

This is my understanding of how it works:
When you set up the curl paths in VS. You're telling VS where to find the curl libraries. But you haven't told C4D (the program) where to find them.
Maxon has set up the c4d program to look for the C4D based .dll files in a specific folder/folders. So that our plugins can make use of them.
The C4D program is basically a plugin reader program.
It grabs the .dlls from where it was told to grab them. The ones that ship with it, and the ones built in and our plugins.
When you bring in an alien .dll file. C4D has not been told where and how to use it.
But when C4D launches. It's able to read a .dll file if it's in the MAXON folder.

As an example. QT is a very popular library people use for GUI stuff.
The various programs they use QT libraries on have no clue what a QT.dll is. QT.dll is an alien file.
So in addition to telling VS how to link the QT libraries to the code they're writing. They also have to put a copy of the .dll's in the source files when they package up their program to send to the users.
If you look at the files that people include in their programs. If it uses a QT based GUI. And the they did not compile QT as static. Then you should see somewhere in the source files a few QT.dll files.
In c4d. The place where these alien .dll files go is the MAXON folder.

I think most of that is correct. But don't take it as gospel.
I'm still learning how things work myself.

-ScottA

Edit- There's probably a way to put the libcurl.dll in your plugin folder rather than the MAXON folder.
And have C4D somehow read it from there. But I have never done it. People like Robert might know how.

Helper

On 06/10/2014 at 13:47, xxxxxxxx wrote:

Umm...hmmm. Confused
I just re-wrote the plugin from scratch. And this time it runs fine if I put the libcurl.dll file in my plugin folder.
I have no idea why it wouldn't run before. So you might not need to put libcurl.dll in the MAXON folder.

Sorry about any confusion Rui.
This is the first time I've fooled around with .dll files in my c4d plugins.

-ScottA

Helper

On 06/10/2014 at 14:10, xxxxxxxx wrote:

You can statically compile libCurl and then link the static lib. That way you do not need to include the dll when you ship your plugin.

Helper

On 06/10/2014 at 14:41, xxxxxxxx wrote:

Could it be that that is what is happening when I compile in Xcode?
My plugin is running perfectly on my Mac, accessing the internet and checking for the file in a protected folder.
And I haven't copied any library into my plugins folder or to the Maxon folder.

Helper

On 06/10/2014 at 15:30, xxxxxxxx wrote:

That sounds like a logical assumption to me.

It's nice that Macs have that set up for you. PC's require most things to be installed by hand.
On the other hand. If someone does all of the work for you. You never learn how to do it yourself.
Which is why Python is both wonderful & evil.

-ScottA

Helper

On 06/10/2014 at 16:37, xxxxxxxx wrote:

But, for the specific case of plugins it is so nice.
I'm having all this trouble because I devised a method to protect my plugins. It is not perfect (no method is) but it makes me more in control and makes life harder for hackers.
It was relatively simple to implement on my Mac. It is a nuisance that it proves to be so complicated on Windows (never really liked Windows very much and this only makes it worse )

Helper

On 07/10/2014 at 07:38, xxxxxxxx wrote:

Out of curiosity.
How are you converting the curl output to a C4D String type result in your Mac version?
The callback method I posted works. But it looks kind of ugly to me. And I'm wondering if there's a simpler way.

-ScottA

Helper

On 07/10/2014 at 07:58, xxxxxxxx wrote:

Here is my code:

  
//   
// access_url.cpp   
// PolyPaintTag   
//   
// Created by Rui Batista on 6/10/14.   
  
#include "access_url.h"   
  
#ifdef _WIN32   
#include <curl.h>   
#endif   
  
#ifndef _WIN32   
#include <curl/curl.h>   
#endif   
  
#include <string>   
  
using namespace std;   
  
// ***************************************************************   
  
struct MemoryStruct {   
     char* memory;   
     size_t size;   
};   
  
// ***************************************************************   
  
static size_t   
WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)   
{   
     size_t realsize = size * nmemb;   
     struct MemoryStruct *mem = (struct MemoryStruct * )userp;   
        
     mem->memory = (char* )realloc(mem->memory, mem->size + realsize + 1);   
     if(mem->memory == NULL) {   
          /* out of memory! */   
          printf("not enough memory (realloc returned NULL)\n");   
          return 0;   
     }   
        
     memcpy(&(mem->memory[mem->size]), contents, realsize);   
     mem->size += realsize;   
     mem->memory[mem->size] = 0;   
        
     return realsize;   
}   
  
// ***************************************************************   
  
bool check_online(string folder,string reg, string username)   
{   
     CURL *curl;   
     CURLcode res;   
        
     char *passcode=new char[usrpass.length()+1];   
     for(long i=0;i<usrpass.length()+1;i++)   
     {   
          passcode[i]=usrpass[i];   
          passcode[i+1]='\0';   
     }   
        
     bool success=true;   
        
     struct MemoryStruct chunk;   
     chunk.memory = (char* ) malloc(1); /* will be grown as needed by the realloc above */   
     chunk.size = 0;    /* no data at this point */   
        
     curl = curl_easy_init();   
     if(curl) {   
          string baseurl="www.my_url.com";   
          string usrpass="username:password";   
             
          char *the_url=new char[baseurl.length()+1];   
          strcpy(the_url, baseurl.c_str());   
             
          curl_easy_setopt(curl, CURLOPT_URL, the_url);   
          curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY);   
          curl_easy_setopt(curl, CURLOPT_USERPWD , passcode);   
          curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);   
          curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void * )&chunk;);   
          curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcurl-agent/1.0");   
             
          /* Perform the request, res will get the return code */   
          res = curl_easy_perform(curl);   
             
          /* always cleanup */   
          curl_easy_cleanup(curl);   
             
          if (chunk.size!=10)   
               success=false;   
          else   
               if(chunk.memory)   
               {   
                    char *b=new char[2];   
                    for(long i=0;i<10;i++)   
                    {   
                         string a=reg.substr(i,1);   
                         b[0]=chunk.memory[i];   
                         if (a!=b) success=false;   
                    }   
                       
                    free(chunk.memory);   
               }   
             
          delete [] the_url;   
             
             
          /* Check for errors */   
          if(res != CURLE_OK) success=false;   
     }   
        
     delete [] passcode;   
        
        
     return success;   
}

I changed a few things to protect my methods of checking the serialization.
However, you end up with a struct containing the bytes that were read and a count of the bytes read.
Assuming the chunk is a struct of type MemoryStruct, I guess it is just a matter of doing something like:

  
char *my_string=new char[chunk.size+1];   
for(long i=0;i<chunk.size;i++)   
{   
     my_string[i]=chunk.memory[i];   
     my_string[i+1]='\0';   
}

Helper

On 07/10/2014 at 08:06, xxxxxxxx wrote:

Ok. Thanks.
You're using the same callback.
I was just curious if you were doing it a different way.

Are you done now?
All problems solved?

-ScottA

Helper

On 07/10/2014 at 08:15, xxxxxxxx wrote:

I had to pause for a while due to "real-life" work
But I was getting all the errors when compiling for Win32.
When I compiled for x86, all errors stopped. I still have to test out.
Just picture this: I finally compiled without the errors and, all of a sudden, the phone rings and a complex and urgent illustration just arrives.
Damn!!! I must finish it to get back to my plugin (I can't have all the required apps opened for the illustration and Windows, at the same time)
Oh, by the way, do I only need to compile for x86?
Otherwise, I will have to install another version of curl also and do some conditional compiling

Helper

On 07/10/2014 at 08:29, xxxxxxxx wrote:

I think you'll need both.

When I tried to use my 32bit version of curl in a 64 bit C4D plugin. That's when I got strange unresolved errors from the init() function.
So I'm guessing that you'll need to have both versions of curl built. And then point to the bits version of curl that matches your C4D plugin bits version.

-ScottA