Your browser does not support the canvas tag.

Note: you can put any alternative content here. -->
Week 11
(although that's a lie. Let's call it Week Awesome)

Margret Hamilton
"Scraping" a website

Scraping mean pulling from the web - it often involves looking through the source file of a website to gra the information.
Any image on the web you can put into your sketch, however, there are three problems with this.

  1. It's called "hot linking" and now a very good etiquette
  2. Someone could easily remove or change the image and it would screw up your sketch
  3. Your sketch will crash it the computer is not on-line.

Awesome hotlinking reverse revenge story
PImage webImg;
void setup() {
  size(400, 400);
  webImg = loadImage("http://www.bampfa.berkeley.edu/images/art/fermata/MFA_mckay.jpg");
}
void draw() {
  image(webImg, 0, 0);
}
You can even grab the HTML itself using laodString();
String [] website; 
void setup() {
  size(400, 400);
  website  = loadStrings("http://faculty.purchase.edu/joseph.mckay");
  for (int i = 0; i < website.length; i ++) {
    println(i + "   " + website [i]);
  }
}
void draw() {
}
Although doing anything interesting with that will require some parsing, which can get messy, and it's easy for someone to change the HTML and mess you up.
There are two ways to get live data from a website into Processing, XLM and Json. Both have their own difficulties and circumventing security issues is tricky on a number of sites. Often, files are saved as Comma Separated Value .csv files. You will have to download the data to your data folder, but it is relatively easy to work with once you do.

RSS and XLM

You can put the rss feed right into an array and use some special tools to parse it.
// We're making a variable for all the text in the XML feed
XML xml;
// and this is the URL we're pulling from. 
String url = "http://www.huffingtonpost.com/section/front-page/feed"; 
XML[] titleXml; 
PFont font; 

void setup() {
  size(800, 300); 
  xml = loadXML(url); // go get the file from the internet
  titleXml = xml.getChildren("channel/item/title");
  /* the line above goes through the all the text in the XML file, finds all the <title> 
   tags that are nested under <item> tags, that are nested under the <channel> tag. So 
   to create a condensed example of what might appear in the actual file...
   <channel>
   <item>
   <title>
   On Golf: For L.P.G.A. Commissioner Mike Whan, Politics Become Part of the Game
   </title>
   </item>
   */
  // And the following loop, using getContent() just goes through the array and pulls 
  // what's between the <title></title> tags and prints it in the console...
  for (int i = 0; i < titleXml.length; i++) {
    println(titleXml[i].getContent());
  }
  font = createFont("helvetica", 12); 
  textFont(font);
}
void draw() {
  background(200); 
  String firstHeadline = titleXml[0].getContent(); 
  fill(100); 
  text(firstHeadline, 20, 100);
}
Try and Catch

Normally if your code only works some of the time you need to fix that problem, but sometimes the bug is not your problem. Try and Catch let's your code keep going even if it throws an error message. This way if the webiste is not responding your program deosn't crash.
XML xml;
String url = "http://www.huffingtonpost.com/section/front-page/feed"; 
XML[] titleXml; 
PFont font; 
boolean goodText; 

void setup() {
  size(800, 300);
  font = createFont("helvetica", 12); 
  textFont(font);
  getText();
}
void getText() {
  try {
    xml = loadXML(url); // go get the file from the internet
    titleXml = xml.getChildren("channel/item/title");
    goodText = true;
  } 
  catch (Exception e) {
    goodText = false;
  }
}
void draw() {
  background(200); 
  if (goodText) {
    String firstHeadline = titleXml[0].getContent(); 
    fill(100); 
    text(firstHeadline, 20, 100);
  } else {
    text("no text loaded try again", 20, 100);
  }
}

void mousePressed() {
  getText();
}
Let's see if we can get the local weather in our program.
http://w1.weather.gov/xml/current_obs/
Uh - I wonder if this works as a twitter rss reader?
IN Class project. Find an RSS or XML feed and see if you can get some data into a sketch. Fun!

TIPS, only call to the website once if you can. Repeatedly polling the page in the draw 60 times a second will backfire - people don't like that.


What is a PVector?

A PVector is a special variable that holds two or three variables within it. Often an objects location will be saved as a single PVector. There is a special subset of commands that you can use on PVectors - which are time savers and can greatly expand your code.
notice that getting at the values in a PVector require using the dot syntax.

int circlesize = 300; 
PVector loc = new PVector(100, 200);
void setup() {
  size(400, 400);
  ellipse(loc.x, loc.y, 100, 100);
}
void draw() {
}

Your browser does not support the canvas tag.


This example uses two vectors, one for velocity, one for location. Use the arrow keys to navigate. If you want to learn more about PVectors in specifics and coding in general there's a great book called The Nature of Code
PVector pos = new PVector(200.0, 200.0); 
PVector posOld = new PVector(200.0, 200.0); 
PVector vel = new PVector(0.0, 0.0); 
float speed = 0;  
float rot = 0; // direction pos is going 
boolean up, down, left, right; 
float decay = .985;
float thrust = 0.05;

void setup() {
  size (600, 600);
  rectMode(CENTER);
  smooth();
}
void draw() {
  background(200); 

  pos.x += vel.x;  // go
  pos.y += vel.y;
  vel.x *= decay; // slow down
  vel.y *= decay;

  if (up == true ) { // add thrust
    vel.x += thrust * cos(rot);
    vel.y += thrust * sin(rot);
  }

  if (left == true) {
    rot -= .05;
  }
  if (right == true) {
    rot += .05;
  }  
  if (down == true) {
    decay = .95;
  }
  else {
    decay = .990;
  }
  
  float rotDrifting = atan2(posOld.y-pos.y, posOld.x- pos.x);
  println(rotDrifting); 
  posOld.x =  pos.x; 
  posOld.y =  pos.y;

  pushMatrix();
  translate(pos.x, pos.y); 
  rotate(rot); 
  if (up == true ) { 
    noStroke();
    fill(250, 250, 60); 
    ellipse(-16, 0, 30, 8);
  }
  stroke(1); 
  fill(120); 
  rect(-10, 0, 8, 10);  
  rect(0, 0, 20, 6); 

  popMatrix();
}

void keyPressed() {
  if (keyCode == LEFT) {
    left = true;
  }  
  if (keyCode == RIGHT) {
    right = true;
  }  
  if (keyCode == UP) {
    up = true;
  }  
  if (keyCode == DOWN) {
    down = true;
  }
}
void keyReleased() {
  if (keyCode == LEFT) {
    left = false;
  }  
  if (keyCode == RIGHT) {
    right = false;
  }  
  if (keyCode == UP) {
    up = false;
  }  
  if (keyCode == DOWN) {
    down = false;
  }
}

The Third Dimmension

Dealing with the 3rd dimension gets tricky. Our screens are inherently 2 dimensional, so we have to "fake" depth. If you look at the drawing with the arrows, the x axis gets larger as it goes to the right, and the y axis gets larger as it goes down (same as in 2d). The Z axis gets larger as it gets "closer" to the screen.
You'll recall using translate and pushMartix/popMatrix when we rotated 2d objects earlier in the semester. This concept is the same when drawing in 3d only everything, even things we don't need to rotate are transformed first. So to draw a box you would
  1. pushMatrix to save the current location of 0,0,0
  2. translate to the location you would like the box to be (don't forget the z axis)
  3. rotate, if you want around either axis.
  4. draw the box a the new location
  5. popMatrix to set the current location back to the original location
  6. repeat this for every object that you want to draw
When you set up a scene in 3d, you need to tell processing in the size command by adding "P3D"

void setup() {
  size(300, 300, P3D);
}
void draw() {
  pushMatrix(); 
  translate(100, 200, 0); 
  rotateX(3.8); 
  box(75); 
  popMatrix();
}

Notice that box only needs one argument, the size, because you always draw a box around the point 0, 0, 0. You can draw regular 2d primitives, and they will still display in 3d as flat planes. Try drawing a scene with a mix of box, sphere, rect, and ellipse.

You can draw more complex shapes with beginShape() and endShape().


void setup() {
  size(640, 360, P3D);
}
void draw() {
  background(0);
  pushMatrix(); 
  translate(width/2, height/2, 0);
  stroke(255);
  float r2 = map(mouseX, 0, width, 0, TWO_PI); 
  rotateX(PI/3);
  rotateZ(r2);
  noFill();

  beginShape();
  vertex(-100, -100, -100);
  vertex( 100, -100, -100);
  vertex(   0, 0, 100);

  vertex( 100, -100, -100);
  vertex( 100, 100, -100);
  vertex(   0, 0, 100);

  vertex( 100, 100, -100);
  vertex(-100, 100, -100);
  vertex(   0, 0, 100);

  vertex(-100, 100, -100);
  vertex(-100, -100, -100);
  vertex(   0, 0, 100);

  endShape();
  popMatrix();
}

"> Lights
There are four different types of lights. Java does not do "Ray Tracing" which is what you need to cast realistic shadows. Sorry.
Look these up in the API. Of course you can iterate any of the values that define a light (location, color, focus). Don't forget to call lights(); to enable them.
You need to put lights in the draw, they cannot just sit in the setup area. Texture
To apply a texture you need to use the beginShape method, you cannot apply a texture directly to a 3D primitive. You need to load the image into a PImage variable like you would a normal image.
Notice in this example the arguments in the beginShape are "normalized". This means they are simplified to -1, 1, and 0. To stretch it use scale();

float rx = 0;
PImage blueCheese; 
void setup() { 
  size(800, 600, P3D); 
  blueCheese = loadImage("blue_cheese.jpg"); 
  textureMode(NORMAL);
}
void draw() {
  background(200); 
  rx += .05; 
  translate(width / 2, height / 2, 0);
  rotateY (rx); 
  stroke(255);
  scale(320, 240, 0);
  beginShape();
  texture(blueCheese); 
  vertex(-1, -1, 1, 0, 0);
  vertex( 1, -1, 1, 1, 0);
  vertex( 1, 1, 1, 1, 1);
  vertex(-1, 1, 1, 0, 1);
  endShape();
}
camera
The "camera" in the 3D space adds another level of complexity, and also control. Think of the camera as the eyes with which we look at the scene. It is not a camera in the traditional sense - it does not "record" anything. If we go back to our list from the top of the page,
  1. put the camera at the location you want
  2. point the camera in the direction you want
  3. rotate the camera around its own axis the way you want
  4. pushMatrix to save the current location of 0,0,0
  5. translate to the location you would like the box to be (don't forget the z axis)
  6. rotate, if you want around either axis.
  7. draw the box a the new location
  8. popMatrix to set the location back to the center point
  9. repeat steps 4 - 8 for each object you want to draw
  10. Normally you would not move the camera more than once during a draw cycle, but there's no rule that says you can't
The camera function has nine parameters with an x y and z value for: where the camera is located, what direction it's looking, and how its titled

camera(width/2, height/2, (height/2) / tan(PI/6), width/2, height/2, 0, 0, 1, 0);


void setup() {
  size(300, 300, P3D);
}
void draw() {
  background(200); 
  pushMatrix(); 
  camera(width/2, height/2, (height/2) / tan(PI/6), mouseX, height/2, 0, 0, 1, 0);
  translate(100, 200, 0); 
  rotateX(3.8); 
  box(75); 
  popMatrix();
}

Manipulating the camera can be confusing - ocd (Obsessive Camera Direction) library is very useful in making it a lot easier, especially if you have multiple camera movements at once.

import damkjer.ocd.*;
Camera camera1;
float turn = 0; 
float r = .1;
float thrust, sidethrust; 
boolean moveleft, moveright, moveforward, movebackward, turnleft, turnright; 

void setup() {
  size(640, 360, P3D);
  fill(204);
  camera1 = new Camera(this, 100, 0, 300);
}

void draw() {
  lights();
  ambientLight(200, 200, 102);
  background(250);
  camera1.feed();// overides the default camera 
  camera1.dolly(thrust);// frontback
  camera1.truck(sidethrust);// strife
  camera1.pan(radians(turn) / 2.0); // turn

  pushMatrix(); 
  translate(width/2, 0, 0); 
  box(40, 70, 40);
  popMatrix();

  pushMatrix(); 
  translate(0, -50, 0); 
  r += .01;
  rotateY(r); 
  box(40, 200, 50);
  popMatrix();

  if (turn > TWO_PI) {
    turn = TWO_PI - turn;
  }
  if (turn < TWO_PI * -1) {
    turn = (TWO_PI * -1) - turn;
  }
  if (moveleft == true) {
    sidethrust = -15;
  }
  else if (moveright == true) {
    sidethrust = 15;
  }
  else {
    sidethrust = 0;
  }
  if (moveforward == true) {
    thrust = -15;
  }
  else if (movebackward == true) {
    thrust = 15;
  }
  else {
    thrust = 0;
  }

  if (turnleft == true) {
    turn = -5;
  }

  else  if (turnright == true) {
    turn = 5;
  }
  else {
    turn =  0;
  }
}

void keyPressed() {

  if (key == 'a') {
    moveleft = true;
  }
  if (key == 'd') {
    moveright = true;
  }
  if (key == 'w') {
    moveforward = true;
  }
  if (key == 's') {
    movebackward = true;
  }

  if (key == 'j') {
    turnleft = true ;
  }
  if (key == 'k') {
    turnright = true;
  }
}

void keyReleased() {
  if (key == 'a') {
    moveleft = false;
  }
  if (key == 'd') {
    moveright = false;
  }

  if (key == 'w') {
    moveforward = false;
  }
  if (key == 's') {
    movebackward = false;
  }
  if (key == 'j') {
    turnleft = false ;
  }
  if (key == 'k') {
    turnright = false;
  }
}

Lab week 12
3D video texture example
Using PGraphics with 3D
Combining a class with 3d
Combining 2 classes with 3d
Daniel Shifman's 3D tutorial from the processing.org site.


importing .csv files

link to csv file

At the heart of parsing a csv file is the command "split". This lets us chop up a string based on the punctuation (in fact we could use any punctuation but commas are common)

void setup() {
  size(400, 400);
  String allAnimals = "dogs, cats, mice"; 
  String [] animaArray = split(allAnimals, ","); 
  println(animaArray[1]);
}
void draw() {
}
Any spreadsheet application can export a .csv file, but you can also get them on the web. For large data sets, getting the data into the correct arrays is a little tricky, but it's totally doable.

In the setup we import all the fields into the string array called allFields
Then we use split inside a for loop to filter out the first name, last name, and major into their own arrays.

In the draw we check the major array and change the color accordingly.

PFont font; 
String [] allFields; 
String [] firstName; 
String [] lastName;
String [] major; 

void setup() {
  size (500, 600); 
  font = loadFont("Dialog-48.vlw");
  textFont(font, 24);

  allFields = loadStrings("class.csv"); 
  firstName = new String [allFields.length] ; 
  lastName = new String  [allFields.length] ; 
  major = new String  [allFields.length] ; 

  for (int i=0; i < allFields.length; i++) {
    String [] person = split(allFields[i], ',');
    lastName [i] = person[0]; 
    firstName [i] = person[1]; 
    major[i] = person[2];
  }
}

void draw() {
  background(20); 
  for (int i = 1; i < allFields.length; i ++) {
    if (major[i].equals("NEW MEDIA") == true) {
      fill(255, 255, 0);
    }
    else if (major[i].equals("GRAPHIC DESIGN") == true) {
      fill(0, 255, 255);
    }
    else if (major[i].equals("SCULPTURE") == true) {
      fill(100, 100, 200);
    }
    else {
      fill(255, 255, 255);
    }
    text( firstName [i] +  " " + lastName [i], 30, i * 30 + 50);
  }
}
This is pretty sweet, but we still needed a .csv file and who has time for that? Well, these guys for one. Yup, you can get .csv data right off the web. There's lots to learn on this site, but their API seems pretty robust and user friendly.
PFont font; 
String [] allFields; 
String [] firstName; 
String [] lastName;
String [] major; 

void setup() {
  size (500, 800); 
  font = loadFont("Dialog-48.vlw");
  textFont(font, 10);
  allFields = loadStrings("https://www.govtrack.us/api/v2/role?current=true&format=csv&fields=person__lastname,person__firstname,party&limit=100");
  println(allFields); 
  firstName = new String [allFields.length] ; 
  lastName = new String  [allFields.length] ; 
  major = new String  [allFields.length] ; 

  for (int i=0; i < allFields.length; i++) {
    String [] person = split(allFields[i], ',');
    lastName [i] = person[0]; 
    firstName [i] = person[1]; 
    major[i] = person[2];
  }
}

void draw() {
  background(20); 
  for (int i = 1; i < allFields.length; i ++) {
    if (major[i].equals("Democrat") == true) {
      fill(255, 255, 0);
    }
    else if (major[i].equals("Republican") == true) {
      fill(0, 255, 255);
    }
    text( firstName [i] +  " " + lastName [i] + " " + major [i], 30, i * 14 + 20);
  }
}
Data sets

data.gov
usGov /
Data visualization of us knowledge